[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Another newbie question

From: WJ Atsma
Subject: Re: Another newbie question
Date: Fri, 1 Aug 2003 10:44:31 +0200


Number of decimal places and precision are not necessarily related, at least not on a computer. The relationship you learn in math and physics (round off to the number of significant digits + 1) is of no concern to the computer. Matlab and excel have the same limitation. The programmer's choice to show zeros rather than the imprecise results from round-off errors after an operation may be nice for the intuitive user, but don't really provide information. For example, 1.000000 +/- 0.2 is just as accurate as 1.00233748 +/- 0.2. The round-off provides useful information
sometimes even.

Not to say that something useful couldn't be done. It should be possible to pass numbers with a precision qualifier and propagate it through your equations, but nobody has done this and I don't know of any other numerical packages/spreadsheets
that do.

Happy computing,


On 2003.08.01 02:02 Fausto Arinos de A. Barbuto wrote:

On Thu, 2003-07-31 at 10:58, John W. Eaton wrote:

> | Take Excel as an example. One can increase
> | the floating point representation of any real number by as many
> | decimal places as he/she wants. However, only zeroes are shown
> | from the 14th decimal place on. It also marvels me that a much-
> | less-than-professional program such as Windows Calculator can
> | represent real numbers with 31 exact decimal places -- and Octave
> | can't.
> If you are trolling, find another venue.  If you have a real

Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:
How to fund new projects:
Subscription information:

reply via email to

[Prev in Thread] Current Thread [Next in Thread]