Re: [gnugo-devel] SlugGo v.s. Many Faces

[Douglas Ridgway]

On Mon, 23 Aug 2004, David G Doshay wrote:

> We have been encouraged by our success against the GNU Go base code, 
> where we can win about 70% of the games when giving GNU Go a 5 stone 
> handicap [...]

> We are now confident that we are indeed much stronger than GNU Go 
> because of our results against Many Faces of Go.

Does anyone happen to know the current strength difference between Gnu Go 
and Many Faces?

> [game results...]
> So, SlugGo wins about 20 out of 25 from Many Faces. We will continue 
> this contest against Many Faces until we have 100 games.

Very interesting. I took the liberty to run some numbers on this data (*).  
The mean score is W+31 (95% CI W+11 - W+51), and the win rate is 78% (95%
CI 56% - 93%). It seems clear that SlugGo is more than 1/2 stone better
than ManyFaces (p = 0.0053 to get this by flipping coins).

Given that, I don't understand the plan to play more games at a handicap
of 0.5. If we take the average margin of 31 as indicative of the handicap
level, and use 2*(komi 6.5) as the value of each stone, we estimate SlugGo
as 2.38+0.5 = 2.88 stones better than Many Faces. If this is right, you
ought to be able to show a statistically significant margin at a 1.5 stone
handicap (2 stones, no komi) within a hundred games, probably less. 3
stones is probably the fair handicap, but it would likely require hundreds
of games to determine who was better at three stones.

So that'd be my vote. Give two stones to Many Faces, or take on another 
top program, or play against people.

doug.

(*) I only see 24 games here, and I threw out the game with the unclear 
result, so I worked with 23 games.