[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Single/double precision equality question
From: |
Daniel J Sebald |
Subject: |
Single/double precision equality question |
Date: |
Fri, 26 Sep 2014 18:34:35 -0500 |
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Fedora/3.1.16-1.fc14 Thunderbird/3.1.16 |
Should the following equality behavior be explained in "help =="?
format bit
single(0.1)
ans = 0011111110111001100110011001100110100000000000000000000000000000
double(0.1)
ans = 0011111110111001100110011001100110011001100110011001100110011010
single(0.1) == double(0.1)
ans = 0011111111110000000000000000000000000000000000000000000000000000
I guess I'm OK with it. The test demotes the double to a single. Try:
double(single(0.1)) == double(0.1)
ans = 0000000000000000000000000000000000000000000000000000000000000000
However, the demoting rule doesn't hold true for all classes. Try:
format
v1 = pi;
v2 = int8(v1);
v1 == v2
ans = 0
Whatever the case, it probably would be good to have this documented for
the user. Perhaps it is, but I don't know where to look beyond "help ."
and "help ==".
Dan
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Single/double precision equality question,
Daniel J Sebald <=