Re: Question about Range class

[Przemek Klosowski]

   As an accuracy test I tried    sqrt(2)*sqrt(2)-2

   My TI calculator came back with...      1e-11     ...error
   octave came back with...             4.44e-16      ...interesting.  

You do know why this is, right? the error in such calculations is on the
order of a Least Significant Digit in whatever numbering system your machine
uses in the calculation. Octave uses double precision IEEE floating point,
which has 52 mantissa binary digits, so the error is on the order of 1/2^52
(octave variable 'eps', equal to ~ 2.220446e-16). TI calculators use a different
floating point format, BCD-based, if I remember correctly--and I'd guess it
uses 11 significant decimal digits.



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------