[Octave-bug-tracker] [bug #30742] Function rem returns incorrect result for certain inputs

[Jaroslav Hajek]

Update of bug #30742 (project octave):

                  Status:                    None => Invalid                
             Open/Closed:                    Open => Closed                 

    _______________________________________________________

Follow-up Comment #1:

No, this is a misunderstanding on your side. The default data type in Octave
is a base-2 floating point number, which can't exactly represent most decimal
numbers (the reason for base 2 is speed).
Hence, when you enter 2e-5, the closest approximation is used instead, and
likewise for 0.1e-5. That's why you don't get exact zero. Try this:

octave:1> 2e-5 / 0.1e-5 - 20
ans =  3.5527e-15

See? You can also try, e.g., in Python to confirm that this is a general
issue and nothing specific to Octave:

Python 2.6.2 (r262:71600, Jun 17 2010, 13:37:45) 
[GCC 4.4.1 [gcc-4_4-branch revision 150839]] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 2e-5 / 0.1e-5 - 20
3.5527136788005009e-15

Unless you can work this around yourself by eliminating the common negative
power of 10, you need a decimal number package. There used to be one on
OctaveForge, but currently it seems unmaintained.


    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?30742>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/