help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Difference double/single precision


From: Jordi Gutiérrez Hermoso
Subject: Re: Difference double/single precision
Date: Wed, 21 Sep 2011 18:07:05 -0500

On 21 September 2011 17:53, Schirmacher, Rolf
<address@hidden> wrote:

> octave.exe> format long
> octave.exe> 1.0/3
> ans =  0.333333333333333
> octave.exe> single(1.0/3)
> ans =  0.333333343267441
> octave.exe> 1.0/3 - single(1.0/3)
> ans = 0
>
> I am wondering about the difference being 0.
> In addition, how could I calculate the difference of 1.0/3 as single and
> double?

The 1.0/3 gets cast to a single when operating with a single. If you
want to see the difference, do

    1.0/3 - double(single(1.0/3))

so you can perform the subtraction as doubles, not singles.

HTH,
- Jordi G. H.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]