[Bug-gnubg] Bug - Multiple errors in rollouts.

I rolled out the same position using three Windows versions of gnubg: 0.14-dev 1.1379, 0.14-dev 1.1399, and 0.14-dev 1.1427. To my dismay, as the version number of gnubg increased, the displayed results were progressively worse! I have attached the rollouts as files: Rollout_1_1379.sgf, Rollout_1_1399.sgf and Rollout_1_1427.sgf

The position in question is a last-ditch hit example, with the proper move bar/23* 6/1 the best way to catch the 2^nd checker. Gnubg plays this move correctly at all plies. The rollouts were 0-ply, 1296 trials, no truncation.

To show the problems, I will include the rollout line for bar/23* 6/1 from each rollout:

Gnubg version Win Win G Win BG Lose Lose G Lose BG Cubeful

1.1379 0.514 0.000 0.000 0.486 0.089 0.041 +0.137

1.1399 0.514 0.000 0.000 0.486 0.000 0.000 +0.114

1.1427 0.515 0.002 0.001 0.485 0.000 0.000 +0.135

Because the best way to win is to close out two checkers, gnubg must break the prime to execute a trap play, if the two checkers make an anchor. Therefore, I expected gnubg to lose some gammons and backgammons if user gets lucky, and version 1.1379 seemed to give reasonable results. Bug #1 - Both of the later versions show zero for the lost gammons and lost backgammons. In addition, the standard error displayed for both lost gammons and lost backgammons is insanely large and increases as the rollout proceeds. Bug #2 - Version 1.1427 (I think the problem happened by October 4, 2003) shows user losing small numbers of gammons and backgammons, after he has borne 13 checkers off!

I did not use the same seeds for the rollouts. I just went backwards in time to earlier versions of gnubg, until I found one I had that was not showing both of these problems.

Bob Eberlein

From:	Eberlein, Robert (REBERLEI)
Subject:	[Bug-gnubg] Bug - Multiple errors in rollouts.
Date:	Wed, 15 Oct 2003 20:24:15 -0400