[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Octave-bug-tracker] [bug #43728] rounding error for integer range multi
From: |
Markus Appel |
Subject: |
[Octave-bug-tracker] [bug #43728] rounding error for integer range multiplied with floating point |
Date: |
Sun, 30 Nov 2014 01:19:30 +0000 |
User-agent: |
Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0 |
URL:
<http://savannah.gnu.org/bugs/?43728>
Summary: rounding error for integer range multiplied with
floating point
Project: GNU Octave
Submitted by: mappel
Submitted on: Sun 30 Nov 2014 01:19:29 AM GMT
Category: Interpreter
Severity: 3 - Normal
Priority: 5 - Normal
Item Group: Incorrect Result
Status: None
Assigned to: None
Originator Name:
Originator Email:
Open/Closed: Open
Discussion Lock: Any
Release: dev
Operating System: GNU/Linux
_______________________________________________________
Details:
In the following example, I would expect the superdiagonal to be 1
everywhere:
octave:16> m1=(-1:3)*0.1 ; m2=(0:4)*0.1 ; m1==m2'
warning: mx_el_eq: automatic broadcasting operation applied
ans =
0 1 0 0 0
0 0 1 0 0
0 0 0 0 0
0 0 0 0 1
0 0 0 0 0
... but it isn't due to a floating point error:
octave:17> m1(4)-m2(3)
ans = 2.7756e-17
Seen on Ubuntu 12.04 with octave 3.8.0 and dev. Matlab R2012b gives the
expected result ( with bsxfun(@eq,m1,m2') instead of m1==m2' ).
Complementary tests:
It seems that something goes wrong when the range extends to negative
integers, I could not reproduce it with positive ranges only:
octave:89> m1=(1:5)*0.1 ; m2=(0:4)*0.1 ; m1==m2'
warning: mx_el_eq: automatic broadcasting operation applied
ans =
0 0 0 0 0
1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
And using a decimal that can be represented exactly also seems to work fine:
octave:94> m1=(-1:3)*0.125 ; m2=(0:4)*0.125 ; m1==m2'
warning: operator -: automatic broadcasting operation applied
ans =
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
0 0 0 0 0
_______________________________________________________
Reply to this item at:
<http://savannah.gnu.org/bugs/?43728>
_______________________________________________
Message sent via/by Savannah
http://savannah.gnu.org/
- [Octave-bug-tracker] [bug #43728] rounding error for integer range multiplied with floating point,
Markus Appel <=