[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-gnubg] Possible error in SSE sigmoid calculation

From: Philippe Michel
Subject: Re: [Bug-gnubg] Possible error in SSE sigmoid calculation
Date: Wed, 21 Dec 2011 14:03:57 +0100 (CET)
User-agent: Alpine 2.00 (BSF 1167 2008-08-23)

On Wed, 21 Dec 2011, Joseph Heled wrote:

I find it hard to believe.

There is a simple test - if the two implementations give the same vale for
the same position, ply 0.

Analyzing and exporting to text a sample match at 0-ply, I get this as a rough measure of number of discrepancies between the following options :

nosse : scalar code without SSE intrinsics
ssecvt : SSE intrinsics with cvtps instruction
ssecvtt : SSE intrinsics with cvttps intruction

% diff nosse.txt ssecvt.txt | wc
   16360   94346  681212
% diff nosse.txt ssecvtt.txt | wc
    2434   12056   90952

The differences between nosse and ssecvtt are due, I think, to the usage of a reciprocal instead of a true division in 1/(1+exp(x)). I suppose this was found acceptable at the time and would dissappear if everything was compiled with the -ffast-math flag. Maybe one could use something like

#ifdef __FAST_MATH__
... use reciprocal intrinsic
... use divide intrinsic

in neuralnetsse.c

The discrepancies in ssecvt, on the other hand, are not only about seven times more common but come from the flawed approximation of exp(x) in addition to the above shortcut.

Looking cursorily at the exports, I found only occasionnal 0.001 differencies in the ssecvtt case, without moves swapped.

In the ssecvt case, there were some almost-equal moves ordered differently (in the exported top five choices). I didn't found cases where the top move was changed or equity differences of more than 0.001, but I'm convinced the former, at least, could happen.

These are still pretty small differences. In the final summary of these analyses for instance I get :

nosse :   Error total EMG (MWC)  -4.142 (-14.676%) -4.845 (-15.479%)
ssecvt :  Error total EMG (MWC)  -4.142 (-14.682%) -4.828 (-15.344%)
ssecvtt : Error total EMG (MWC)  -4.143 (-14.683%) -4.846 (-15.484%)

reply via email to

[Prev in Thread] Current Thread [Next in Thread]