octave-maintainers
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: integer arithmetics


From: John W. Eaton
Subject: Re: integer arithmetics
Date: Thu, 02 Oct 2008 15:12:06 -0400

On  2-Oct-2008, Jaroslav Hajek wrote:

| On Thu, Oct 2, 2008 at 9:30 AM, Jaroslav Hajek <address@hidden> wrote:
| > On Thu, Oct 2, 2008 at 7:47 AM, Jaroslav Hajek <address@hidden> wrote:
| >> On Wed, Oct 1, 2008 at 10:36 PM, John W. Eaton <address@hidden> wrote:
| >
| >>
| >> I'll resend an update.
| >>
| >
| >
| > And here it is.
| >
| 
| Sorry, here it is. It appears to work reasonably well even if I
| undefine HAVE_FAST_INT_OPS and SIZEOF_LONG_DOUBLE. I fixed a few
| places and modified multiplication and division to get correct
| behavior in the common cases like 2*a or a/3.
| If you want the illustration of the remaining problems, try:
| ((int64(2**62) + 2) * 0.5) - (int64(2**61) + 1)
| which should, in theory, give zero. These are really corner cases, as
| nobody would probably do a*0.5 and intend integer arithmetics, but
| still an inconsistency. I think we may warn about it in the manual. If
| you replace * 0.5 with / 2, it should work (though it's still division
| by a double).
| 
| Nothing like this should happen for addition and subtraction - try
| (3*2**62 + (1+intmin('int64'))) == (int64(2**62)+1)
| or
| (3*2**63 - intmax('uint64')) == (uint64(2**63)+1)
| 
| These should both be true (note that in both cases the first operand
| is outside the integer range).

It seems bad to get incorrect answers.  I'd guess people are likely to
think that multiplying by 0.5 is the same as dividing by 2.

Would it be better to skip mixed-type operations for int64?  Or at
least skip the ones that can cause trouble?

jwe


reply via email to

[Prev in Thread] Current Thread [Next in Thread]