[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #61041] Sum behaves unexpectedly for large uin
|
From: |
Markus Mützel |
|
Subject: |
[Octave-bug-tracker] [bug #61041] Sum behaves unexpectedly for large uint64 values when not using native mode |
|
Date: |
Sun, 15 Aug 2021 12:32:36 -0400 (EDT) |
|
User-agent: |
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36 Edg/92.0.902.67 |
Update of bug #61041 (project octave):
Status: None => Invalid
Open/Closed: Open => Closed
_______________________________________________________
Follow-up Comment #1:
`flintmax`, i.e. the maximum integer up to which all integers can be
represented accurately in floating point precision, is 2^53 (about 9e15) for
double precision floating point numbers. If the result of the sum is in the
order of 4e17 (i.e. approx. 2 orders of magnitude higher than `flintmax`),
relatively large inaccuracies are to be expected in general.
On the other hand, the maximum number that can be represented in uint64 is
2^64-1 (about 1.8e19). So performing a native sum should work correctly.
Closing as invalid.
Please, comment if you think that Octave should do something different and we
can re-open.
_______________________________________________________
Reply to this item at:
<https://savannah.gnu.org/bugs/?61041>
_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/