octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Single/double precision equality question


From: Daniel J Sebald
Subject: Single/double precision equality question
Date: Fri, 26 Sep 2014 18:34:35 -0500
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Fedora/3.1.16-1.fc14 Thunderbird/3.1.16

Should the following equality behavior be explained in "help =="?

format bit
single(0.1)
ans = 0011111110111001100110011001100110100000000000000000000000000000
double(0.1)
ans = 0011111110111001100110011001100110011001100110011001100110011010
single(0.1) == double(0.1)
ans = 0011111111110000000000000000000000000000000000000000000000000000

I guess I'm OK with it.  The test demotes the double to a single.  Try:

double(single(0.1)) == double(0.1)
ans = 0000000000000000000000000000000000000000000000000000000000000000

However, the demoting rule doesn't hold true for all classes.  Try:

format
v1 = pi;
v2 = int8(v1);
v1 == v2
ans = 0

Whatever the case, it probably would be good to have this documented for the user. Perhaps it is, but I don't know where to look beyond "help ." and "help ==".

Dan



reply via email to

[Prev in Thread] Current Thread [Next in Thread]