help-octave
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

floating point arithmetic problems


From: AG
Subject: floating point arithmetic problems
Date: Tue, 24 Oct 2017 13:27:17 -0700 (MST)

I thought I understood floating point problems but I guess I don't and now I
have a lot of code that is acting strangely given my understanding. 
EG.
0.2-0.1==0.1
ans = 1    #GREAT!

1.2-1.1==0.1
ans = 0    #HUH?

Here's one concrete example from some code I was playing with. all this code
does is rebuilding the decimal column using start:increment:end syntax and
compare the old column to the new.  

test = [
96.02,3827;
96.03,5341;
96.04,6107;
96.05,7134;
96.06,8706;
96.07,12367;
96.08,13971;
96.09,16900;
96.10,15700;
96.11,18791;
96.12,22168;
96.13,21973;
96.14,21800;
96.15,19626;
96.16,17601;
96.17,16496;
96.18,14921;
96.19,10819;
96.20,9087;
96.21,6056;
96.22,5071;
96.23,4311];
newfirstcol = (min(x):0.01:max(x))'; #should have same values as tests 1st
column..
sum(ismember(newfirstcol,test(:,1))) #matches only 16 times....

Is there a simple and consistent way to avoid problems like these? I would
have though that with only 2 decimal places floating point problems(which is
what I assume this is) wouldn't be a problem. Thanks for the help.



--
Sent from: http://octave.1599824.n4.nabble.com/Octave-General-f1599825.html



reply via email to

[Prev in Thread] Current Thread [Next in Thread]