[Bug-gnubg] Bug: 1-ply outperforming 2-ply

[Philippe Michel]

> I've tried swapping the players, using different dice generators
> (random.org, MT), but the 1-ply player almost ALWAYS beats the 2-ply
> player on my machine, by A LOT, even winning by a gammon a fair amount
> of the time. Very strange. Shouldn't the 2-ply player be a lot better
> than the 1-ply player? I've tried running with & without NN pruning,
> move filters, etc, but I always get the same result! What's going on?

In the match attached to your mail, if we analyze it at 1-ply, we get that 
1-ply played perfectly (obviously) and 2-ply dropped about 4% winning 
chances through its few errors. If we analyze it at 2-ply, this is more or 
less the opposite. Each analysis concludes that its level outplayed the 
other and was approximately a 52% favorite.

If this match is typical and the numbers are accurate (we can't know with 
such a superficial analysis, but they look reasonable), hundreds or 
thousands of trials would be needed to know with any reasonable certainty 
which side is stronger. For instance, if you played 10 such matches and 
one side won 8 of them, the chances of it being the strong one is about 2 
to 1 only.

The best evidence available, testing the various levels against the 
Depreli benchmark, is that 1-ply make about twice as many errors as 2-ply.

I'm afraid you simply don't have enough evidence for your claim :-). How 
many matches did you run ? What were the results ? On the other hand I 
could check that the 1-ply and 2-ply moves in your log match what would be 
played on my machine so there is not issue specific to yours.