[Octave-bug-tracker] [bug #61290] Unexpected rounding of min and max with integer types

[Rik]

Follow-up Comment #5, bug #61290 (project octave):

For comment #4, Matlab returns 255 of class uint8.

This is normal behavior for Matlab and Octave.  When mixing integers and
floating point values the result is an integer.

For example,


octave:1> x = uint8 (1) + double (1.0)
x = 2
octave:2> class (x)
ans = uint8


In the test case


max (uint8(250), sqrt(flintmax))


the value "sqrt(flintmax)" is ~9.5E7 and


uint8 (9.5E7) = 255



    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/bugs/?61290>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/