[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [bug-mes] mes lib: dtoab

From: Danny Milosavljevic
Subject: Re: [bug-mes] mes lib: dtoab
Date: Thu, 14 Mar 2019 22:48:16 +0100

Hi Janneke,

the reason that dtoab doesn't work is an overflow in the long.  Did it work on 

IEEE 754 double precision has a 52 bit mantissa which means such a number would 
have 16 significant digits.  That cannot fit into long (10 significant digits), 
so the way dtoab is implemented is not correct.

(It's not as bad as it could be since dtoab handles the fractional part and the 
integral part separately)

In any case, maybe it's enough for our purposes.

The fix for ARM would be:

--- a/lib/mes/dtoab.c
+++ b/lib/mes/dtoab.c
@@ -29,7 +29,11 @@ dtoab (double d, int base, int signed_p)
   long i = (long) d;
   char *p = ntoab (i, base, signed_p);
   strcpy (dtoa_buf, p);
+#if __arm__
+  long f = (d - (double) i) * (double) 100000000;
   long f = (d - (double) i) * (double) 100000000000;
   if (f)
       if (f < 0)

If it was all right with you, we could just unconditionally make the precision 
even worse and just always use the shorter multiplicant :)

Otherwise we would have to have two different expected results in the tests and 
that's not nice.

Furthermore, I don't understand how it worked on i386, if it did.  Because 
sizeof(long) is 4 on i386 and 4 on arm.

Attachment: pgpAiNNPk8K21.pgp
Description: OpenPGP digital signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]