Re: [Bug-gnubg] Possible error in SSE sigmoid calculation

[Joseph Heled]

Hi,

I am not sure which nets and which sigmoid implementation is used by
gnubg now - but note that my nets were trained against an
approximation. It does not matter that much if you use the exact
definition - what is important is using a "valid pair", that is a net
and the sigmoid implementation it was trained with.

In the past, breaking this meant a serious drop in move quality. I
suggest someone checks this ....

-Joseph

On 22 December 2011 02:03, Philippe Michel <address@hidden> wrote:
> On Wed, 21 Dec 2011, Joseph Heled wrote:
>
>> I find it hard to believe.
>>
>> There is a simple test - if the two implementations give the same vale for
>> the same position, ply 0.
>
>
> Analyzing and exporting to text a sample match at 0-ply, I get this as a
> rough measure of number of discrepancies between the following options :
>
> nosse : scalar code without SSE intrinsics
> ssecvt : SSE intrinsics with cvtps instruction
> ssecvtt : SSE intrinsics with cvttps intruction
>
> % diff nosse.txt ssecvt.txt | wc
>   16360   94346  681212
> % diff nosse.txt ssecvtt.txt | wc
>    2434   12056   90952
>
> The differences between nosse and ssecvtt are due, I think, to the usage of
> a reciprocal instead of a true division in 1/(1+exp(x)). I suppose this was
> found acceptable at the time and would dissappear if everything was compiled
> with the -ffast-math flag. Maybe one could use something like
>
> #ifdef __FAST_MATH__
> ... use reciprocal intrinsic
> #else
> ... use divide intrinsic
> #endif
>
> in neuralnetsse.c
>
> The discrepancies in ssecvt, on the other hand, are not only about seven
> times more common but come from the flawed approximation of exp(x) in
> addition to the above shortcut.
>
> Looking cursorily at the exports, I found only occasionnal 0.001
> differencies in the ssecvtt case, without moves swapped.
>
> In the ssecvt case, there were some almost-equal moves ordered differently
> (in the exported top five choices). I didn't found cases where the top move
> was changed or equity differences of more than 0.001, but I'm convinced the
> former, at least, could happen.
>
> These are still pretty small differences. In the final summary of these
> analyses for instance I get :
>
> nosse :   Error total EMG (MWC)  -4.142 (-14.676%) -4.845 (-15.479%)
> ssecvt :  Error total EMG (MWC)  -4.142 (-14.682%) -4.828 (-15.344%)
> ssecvtt : Error total EMG (MWC)  -4.143 (-14.683%) -4.846 (-15.484%)