|
From: | Carnë Draug |
Subject: | [Octave-bug-tracker] [bug #47469] sizemax() is off by one |
Date: | Sat, 19 Mar 2016 19:08:00 +0000 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0 Iceweasel/38.6.1 |
URL: <http://savannah.gnu.org/bugs/?47469> Summary: sizemax() is off by one Project: GNU Octave Submitted by: carandraug Submitted on: Sat 19 Mar 2016 19:07:59 GMT Category: Octave Function Severity: 3 - Normal Priority: 5 - Normal Item Group: Incorrect Result Status: None Assigned to: None Originator Name: Originator Email: Open/Closed: Open Discussion Lock: Any Release: dev Operating System: Any _______________________________________________________ Details: The function sizemax() is off by one. octave> sizemax () ans = 2147483646 octave> intmax ("int32") ans = 2147483647 octave> uint64 (numel (ones (1, 2147483647))) ans = 2147483647 The fact that I can create that array of ones is a separate bug (see bug #47468). The reason for being off by one is mentioned in the source of dim-vector: http://hg.savannah.gnu.org/hgweb/octave/file/2cdbae31c022/liboctave/array/dim-vector.cc#l32 Checking the logs for a better explanation brings up this: http://hg.savannah.gnu.org/hgweb/octave/rev/6683f0c9d742 I removed the "-1" from dim_vector::dim_max and make check reports no new failures. Maybe it is no longer needed (or maybe it's some untested codepath. Anyone has any insight? _______________________________________________________ Reply to this item at: <http://savannah.gnu.org/bugs/?47469> _______________________________________________ Message sent via/by Savannah http://savannah.gnu.org/
[Prev in Thread] | Current Thread | [Next in Thread] |