RE: precision of floor(a/b)

[CdeMills]

Once again, the problem is ill-defined. Knowing that 0.1 and 0.6 don't have a
finite-length representation in binary, have a look for instance in Knuth's
books , which suggest to replace the test on the real numbers
a == b
by
abs(a-b) < eps

this way, you can verify that
abs(6*.1 - .6) < eps

while 
 6*.1 - .6 = 1.1e-16

To avoid implementation-dependent problems, I suggest to compute your loop
as
alpha_min = 0;
alpha_max = 6;
alpha_step = 0.1; 
for alpha = alpha_min : alpha_max,
        # <do some stuff with alpha*alpha_step> 
endfor
This way, the latest processed value will be 6*0.1, and not either .5 or .6,
depending on the way the equality test is performed. It's less elegant,
requires one more multiplication or variable, but it's predictable.

Regards

Pascal
-- 
View this message in context: 
http://n4.nabble.com/precision-of-floor-a-b-tp1752272p1752666.html
Sent from the Octave - General mailing list archive at Nabble.com.