[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Question about Range class
From: |
Robert A. Macy |
Subject: |
Re: Question about Range class |
Date: |
Fri, 25 Nov 2005 12:59:07 -0800 |
Interesting. Didn't know that.
- Robert -
On Fri, 25 Nov 2005 15:29:51 -0500 (EST)
Przemek Klosowski <address@hidden> wrote:
>
> As an accuracy test I tried sqrt(2)*sqrt(2)-2
>
> My TI calculator came back with... 1e-11
> ...error
> octave came back with... 4.44e-16
> ...interesting.
>
> You do know why this is, right? the error in such
> calculations is on the
> order of a Least Significant Digit in whatever numbering
> system your machine
> uses in the calculation. Octave uses double precision
> IEEE floating point,
> which has 52 mantissa binary digits, so the error is on
> the order of 1/2^52
> (octave variable 'eps', equal to ~ 2.220446e-16). TI
> calculators use a different
> floating point format, BCD-based, if I remember
> correctly--and I'd guess it
> uses 11 significant decimal digits.
>
-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.
Octave's home on the web: http://www.octave.org
How to fund new projects: http://www.octave.org/funding.html
Subscription information: http://www.octave.org/archive.html
-------------------------------------------------------------
- Re: Question about Range class, (continued)
- Re: Question about Range class, Pedro Antonio Fructuoso Merino, 2005/11/23
- Re: Question about Range class, Robert A. Macy, 2005/11/23
- Re: Question about Range class, Shai Ayal, 2005/11/24
- Re: Question about Range class, John W. Eaton, 2005/11/24
- Re: Question about Range class, Joe Koski, 2005/11/24
- Re: Question about Range class, Miroslaw Kwasniak, 2005/11/25
- Re: Question about Range class, Robert A. Macy, 2005/11/25
- Re: Question about Range class, Przemek Klosowski, 2005/11/25
- Re: Question about Range class,
Robert A. Macy <=
- Re: Question about Range class, Henry F. Mollet, 2005/11/25
- Re: Question about Range class, Robert A. Macy, 2005/11/25