[Octave-bug-tracker] [bug #61319] idivide fails to distinguish between ceil and floor for inputs close to perfect squares

[Arun Giridhar]

URL:
  <https://savannah.gnu.org/bugs/?61319>

                 Summary: idivide fails to distinguish between ceil and floor
for inputs close to perfect squares
                 Project: GNU Octave
            Submitted by: arungiridhar
            Submitted on: Sat 09 Oct 2021 08:55:58 AM EDT
                Category: Octave Function
                Severity: 3 - Normal
                Priority: 5 - Normal
              Item Group: Incorrect Result
                  Status: None
             Assigned to: None
         Originator Name: 
        Originator Email: 
             Open/Closed: Open
                 Release: dev
         Discussion Lock: Any
        Operating System: GNU/Linux

    _______________________________________________________

Details:

The idivide function gives the same results for ceil and floor options for
certain inputs close to perfect squares, even though that violates the
arithmetic bounds for ceil and floor:

Test:

% test idivide
inputs = [  uint64(4e16) + (-4:+4), ...
            uint64(9e16) + (-8:+8), ...
            uint64(299792458) ^ 2 + (-12 : +4), ...
            uint64(16e18) + (-1024 : +1024), ...
         ];

function testfun (foo)
  bar = uint64 (round(sqrt(foo))) + (-1:1);
  q1 = idivide (foo, bar, "ceil");
  q2 = idivide (foo, bar, "floor");
  q1 == q2     # should be [0 0 0] or [0 1 0]
endfunction

for n = inputs(:)'
  testfun (n); # incorrectly returns [1 1 1]
end


The last line from the test should return [0 1 0] if the input is a perfect
square, or [0 0 0] if it is not. Instead, for certain inputs close to perfect
squares, we get this impossible result: 

Result:

foo = 40000000000000000
bar =
  199999999  200000000  200000001

q1 =
  200000001  200000000  199999999

q2 =
  200000001  200000000  199999999

ans =
  1  1  1






    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61319>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/