[Octave-bug-tracker] [bug #61143] sum of single precision numbers hits precision limit due to naive summation

[Nicholas Jankowski]

Update of bug #61143 (project octave):

                  Status:             In Progress => Confirmed              
                 Summary: Functions sum and mean returns wrong answer for
single precision input => sum of single precision numbers hits precision limit
due to naive summation

    _______________________________________________________

Follow-up Comment #34:

this issue came up again when discussing single precision results in mean over
in bug #63848. That bug is likely going to just focus on making mean avoid the
single issue by processing as double. 

Noting from comment #25, however, that matlab implemented more accurate block
based summation across all data types in 2020 after doing so for single in
2017 (not that we know their block count or routine), it has been suggested
that we may want to implement @mleitner's proposed summation algorithm, or
something similar, in octave as there are now certain summations/means/etc
cases where matlab produces accurate sums and octave does not.  

retitling to focus on sum. leaving single in the title even though an improved
sum algorithm would apply to all data types. reverting to Confirmed as there
isn't currently any active progress on turning @mleitner's code into a patch.


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61143>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/