[Octave-bug-tracker] [bug #61319] idivide fails to distinguish between ceil and floor for inputs close to perfect squares

[Michael Leitner]

Follow-up Comment #8, bug #61319 (project octave):

@mmuetzel: I think that whether the inputs are larger than flintmax or not is
not the only point -- of course, if you have an input that cannot be
represented as double and you cast it to double, it will be complicated to get
it right, but the initial case of foo = 40000000000000000 is exactly
representable (it is only a product of a large number of fives, which is
representable, and the twos in the factorization end up in the exponent). No,
the point is that in this case the double division gives as result (the double
representation of) an integer, which does not change with round, ceil or
floor. So using double division also does not work due to this point, and I
can suspect that this can happen also for inputs smaller than flintmax. 

And I really do think that it is better to test for c*b==a rather than
subtracting the result of mod, if only because an integer multiplication
should be much faster than an integer division (or the modulo operation). I
would think that something along the lines of c+=(c*b>a)*sign(x) should work
(for each mode a different expression), where x is one of a, b or c (I would
have to work it out on paper).

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61319>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/