[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Axiom-developer] i-output and decimal width
From: |
Stephen Wilson |
Subject: |
[Axiom-developer] i-output and decimal width |
Date: |
03 Aug 2007 15:03:49 -0400 |
User-agent: |
Gnus/5.09 (Gnus v5.9.0) Emacs/21.4 |
*,
There was some discussion a while ago about the rounding behaviours of
GCL w.r.t base 10 logarithms, and the impact it has in counting the
number of digits in a radix 10 expansion.
We have the following code in i-output.boot.pamphlet:
u < MOST_-POSITIVE_-LONG_-FLOAT => 1+negative+FLOOR ((LOG10 u) + 0.0000001)
-- Rough guess: integer-length returns log2 rounded up, so divide it by
-- roughly log2(10). This should return an over-estimate, but for objects
-- this big does it matter?
FLOOR(INTEGER_-LENGTH(u)/3.3)
Here, u is a positive integer. The first case which tests against a
bound on most-positive-long-float is almost OK but it adjusts the call
to LOG10 by a small delta which is not enough on some machines.
The final case using integer-length is just not right. The algorithm
returns an extra `digit' to account for a minus sign, which is missed
by this calculation, and it also returns an over estimation.
Could I suggest the following to replace these two cases:
(+ 1 |negative| (round (log n 10)))
- [Axiom-developer] i-output and decimal width,
Stephen Wilson <=