[Octave-bug-tracker] [bug #61319] idivide fails to distinguish between ceil and floor for inputs close to perfect squares

[Arun Giridhar]

Follow-up Comment #1, bug #61319 (project octave):

The lines from scripts/general/idivide.m which cause the incorrect rounding
are here:


    106   if (strcmp (op, "fix"))
    107     z = cast (fix (double (x) ./ double (y)), typ);
    108   elseif (strcmp (op, "round"))
    109     z = x ./ y;
    110   elseif (strcmp (op, "floor"))
    111     z = cast (floor (double (x) ./ double (y)), typ);
    112   elseif (strcmp (op, "ceil"))
    113     z = cast (ceil (double (x) ./ double (y)), typ);
    114   else
    115     error ('idivide: unrecognized rounding type "%s"', op);
    116   endif


Should it be done so that the numerator is rounded up while the denominator is
rounded down? Will it break Matlab compatibility if this is changed?

Also, why are we going through floating point for integer division?

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61319>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/