Re: Bug in rollout ?

[Philippe Michel]

On Wed, Sep 23, 2020 at 06:22:28PM +0200, Øystein Schønning-Johansen wrote:

> I strongly believe that this is a pretty simple position to play

For a neural net this is not so clear. With 7 men stacked on the 7 point 
this is a position that will never happen in real play and is unlikely 
in training. Neither will its immediate continuations. A plausible 
adversarial example indeed.

If you look at the temperature map you will see how badly gnubg plays 
many rolls on the first turn.

Using a one-sided bearoff database going to the 7 point instead of the 6 
helps (and going to the 11 point should totally eliminate the issue), 
but another issue appears : the transition from race net evaluation to 
bearoff database evaluation. If the former overvalues the position and 
its close continuations, X will tend not to move the checker from the 11 
point further than the 8 point to stay with the more generous evaluator.

> I think that X (on roll) will lose about 15% gammon. It should not be 
> hard to roll this out.

This is much less than that. Theodore in another follow-up got a quick 
estimate of 2%. Since O needs doublets but X needs not getting them, one 
can replace his 1/6 factor by 5/36 and get a slightly more accurate 
number of about 1%, showing that your 2 ply rollout is much better than 
0 ply but still far from perfect.

This is arguably a bug, but not a simple coding error : one (of 
undoubtly many) weak spot in evaluations.