[Octave-bug-tracker] [bug #65712] sort over InF dimension coredumps

[Arun Giridhar]

Follow-up Comment #3, bug #65712 (group octave):

I backed it out while this is being dug into.

So on Clang, this is what I found:

* If I remove the line with nint_value and rely only on strict_int_value, then
inf input causes it to try to occupy all memory. So I left the nint_value line
in place as it was.

* If I remove the first line about is_scalar_value, then it erroneously allows
vector inputs for dim, so that line is needed too.

* I added the strict_int_value check to stop fractional dimensions. Then I
added BISTs, verified it all worked on both branches, and pushed. It promptly
failed on a different compiler.

Which of the many integer conversion functions guard against inf inputs?
Evidently nint_value does that only for some compilers. (?!)


    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?65712>

_______________________________________________
Message sent via Savannah
https://savannah.gnu.org/