|
From: | Richard Henderson |
Subject: | Re: [PATCH] utils: Reduce chance of rounding inaccuracy in qemu_strtosz. |
Date: | Sat, 13 Mar 2021 15:48:43 -0600 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 |
On 3/11/21 2:07 PM, Eric Blake wrote:
+ /* + * Add in a fudge-factor (2^53 when double is IEEE format) for + * all scales less than P (2^50), so that things like + * 12.345M with unit 1000 produce 12345000 instead of + * 12344999. + */ + if (mul > 1e49) {
The comment says less than, the code says greater than.
An alternative patch might be writing (uint64_t)(fraction * mul + 0.5) (that is, introduce the fudge factor after the multiplication instead of before). Preferences?
I think I would prefer this, or for further rounding error reduction, fma(fraction, mul, 0.5).
r~
[Prev in Thread] | Current Thread | [Next in Thread] |