[Octave-bug-tracker] [bug #54572] int64 does not saturate correctly in negative direction

[Dan Sebald]

Follow-up Comment #26, bug #54572 (project octave):

OK...

Here's a little patch to proof-of-concept the use of the GNU-lib builtins.  It
only works for int64 because I hard-coded the use of
__builtin_saddll_overflow(), but if you could figure out a way to use the
template T to substitute the proper builtin routine (conditioned on the GCC
compiler being used), there couldn't be any faster routine.

It works, and I think I have the < 0 (versus <= 0) correct, but there should
be plenty of corner cases added to the BIST:


octave:1> intmin ('int64')
ans = -9223372036854775808
octave:2> intmin ('int64') - 1
ans = -9223372036854775808
octave:3> intmin ('int64') + 1
ans = -9223372036854775807
octave:4> intmin ('int64') + intmin ('int64')
ans = -9223372036854775808
octave:5> intmin ('int64') - intmin ('int64')
ans = 0
octave:6> intmin ('int64') - intmax ('int64')
ans = -9223372036854775808
octave:7> intmax ('int64') + intmax ('int64')
ans = 9223372036854775807
octave:8> intmax ('int64') - intmin ('int64')
ans = 9223372036854775807


BTW, something else that would be nice is a special type/class variable
definition that exists only in that intermediate world of the interpreter;
call it a "long_double".  Rather than ASCII numbers being immediately
converted to double they could have the significand stored as a long 64 bit
while the exponent is stored in a double as 1eX.  So if the user types


octave:7> int64(-9223372036854775806)
ans = -9223372036854775808


the number inside the function argument list would be treated as

(long_double) -9223372036854775806 x 1e0

and with the proper construct the int64() routine could convert the
long_double back to a long without loss of resolution.  It would only be when
an assignment is made that the double_long gets converted to double.  (I.e.,
the user would never be able to create or see any type of class called
"double-long".)

(file #44893)
    _______________________________________________________

Additional Item Attachment:

File name: int64_overflow_proof_of_concept.diff Size:1 KB


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?54572>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/