Re: [Bug-apl] Fuzzy Floor and Ceiling

[Juergen Sauermann]

Hi Mike,

page 58 merely says that ⌈ and ⌊ use comparison tolerance, but not in which way.
And page 133 (floor) says: "using comparison" tolerance and not "using RFUZZ".

I read page 58 like this:

a) ⎕CT is the same as comparison tolerance
b) RFUZZ is N×⎕CT  where N is the (magnitude-wise) larger of the numbers involved in a comparison.

Now, the question is if for we should use ⎕CT or RFUZZ when computing ⌊B.

Apparently the IBM APL2 implementation is: B is rounded to the nearest integer N if B=N is 1
The ISO standard and GNU APL first compute D←∣B-⌊B and round up if D=1 is 1.

In other words, the "largest number" used to compute RFUZZ is B in IBM APL2 and 1 in the ISO standard.

What I do not at all like in the IBM APL2 implementation is that ⌊B can be much larger than B which I
find very counter-intuitive..

Now, both are reasonable approaches. But since the ISO standard (which was also written by IBM) explicitly
says how to compute ⌊ and ⌈ (and the IBM language reference, as I read it, does not contradict), I believe it is
better to follow the standard than the IBM APL2 implementation.

/// Jürgen

On 08/14/2015 08:54 PM, Mike Duvos wrote:

Hi Jürgen,

> Z← R For real numbers, yields the largest integer that does not exceed R (within the comparison tolerance).

> That is essentially what ISO says and what GNU APL has implemented. The observed behavior
> of the IBM APL2 implementation seems to be (within R times the comparison tolerance),
> which is probably unintentional.

On page 58 of the APL2 Language Reference, it defines what "within comparison tolerance" means and for two positive numbers, the fuzz factor is ⎕CT times the larger of the magnitudes.  It then lists floor and ceiling as two of the many functions to which this applies.

I cannot think of a single APL that doesn't relativize ⎕CT for floor and ceiling, with the exception of this one.

It's not a problem that relative tolerance can include more than one integer in the interval as the magnitude increases, as you are only checking if you are tolerantly equal to the nearest integer, which always does the right thing.

"within comparison tolerance" should be read as "according to the previously discussed comparison tolerance algorithm" and not taken as a mathematical statement about adding or subtracting ⎕CT,  or contrasted with "within R times comparison tolerance."

Just as if they had said "modulo comparison tolerance," it would not imply anything having to do with residue.

If tolerant ⌊ and ⌈ are only going to do something special on 2049 distinct integers, why not just do it without ⎕CT on everything, and call it a "feature." 

Regards,

Mike