[Qemu-devel] [PATCH] softfloat: Fix factor 2 error for scalbn on denormal inputs

[Peter Maydell]

If the input to float*_scalbn() is denormal then it represents
a number 0.[mantissabits] * 2^(1-exponentbias) (and the actual
exponent field is all zeroes). This means that when we convert
it to our unpacked encoding the unpacked exponent must be one
greater than for a normal number, which represents
1.[mantissabits] * 2^(e-exponentbias) for an exponent field e.

This meant we were giving answers too small by a factor of 2 for
all denormal inputs.

Note that the float-to-int routines also have this behaviour
of not adjusting the exponent for denormals; however there it is
harmless because denormals will all convert to integer zero anyway.

Signed-off-by: Peter Maydell <address@hidden>
---
Found this while going through and fixing "wrong exception flags"
bugs in the 32 bit ARM VCVT instructions. Error is most obvious for
scale factor of n==0, in which case we should return the same value
we were passed, but were halving it instead. Since all denormals
are so small they round to zero in the second half of the fixpoint
conversion the only effect for ARM was erroneously setting Inexact
if the input had a 1 in the least significant bit of the mantissa
and the scale factor was n==0.

This patch is licensed under either the softfloat -2a or -2b licenses,
at your option.

 fpu/softfloat.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index dbda61b..db77b07 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -6795,10 +6795,13 @@ float32 float32_scalbn( float32 a, int n STATUS_PARAM )
         }
         return a;
     }
-    if ( aExp != 0 )
+    if (aExp != 0) {
         aSig |= 0x00800000;
-    else if ( aSig == 0 )
+    } else if (aSig == 0) {
         return a;
+    } else {
+        aExp++;
+    }
 
     if (n > 0x200) {
         n = 0x200;
@@ -6828,10 +6831,13 @@ float64 float64_scalbn( float64 a, int n STATUS_PARAM )
         }
         return a;
     }
-    if ( aExp != 0 )
+    if (aExp != 0) {
         aSig |= LIT64( 0x0010000000000000 );
-    else if ( aSig == 0 )
+    } else if (aSig == 0) {
         return a;
+    } else {
+        aExp++;
+    }
 
     if (n > 0x1000) {
         n = 0x1000;
-- 
1.8.5