Re: [gnugo-devel] Re: Ranked Robot (fwd)

[aquarius]

Hi,

sorry for my late answer but I discovered problems with
GoGui's TwoGtp and had to run part of the tests again
1) it would not recognize the komi it gets from GoGui
   -> gnugo didn't know komi was negative
2) it could not cope with resigning
   -> games were not counted in statistics due to "errors"

below are results for case 4 , 9x9 and 19x19.

Gunnar Farnebäck wrote:
> aquarius wrote:
> > I'm running some tests now for case 4, no handicap, on 9x9 and 19x19.
> > A first glimpse at the results shows that on 9x9 GNU Go does better
> > without the patch (8 wins out of 12 for white so far) than with it
> > (4 wins out of 12).
>
> I don't think this patch will ever have any effect at all on 9x9 for
> case 4. It would be very unusal to have 5x5 vertices around the middle
> point empty after ten moves. :-)

You are right!
I just started the tests without having a closer look at the patch - sorry.
After 100 9x9 games W: 3.5.10          vs.  B: 3.5.10 mirroring
and   100 9x9 games W: 3.5.10 patched  vs.  B: 3.5.10 mirroring
I got these results:
Not patched: 44 wins out of 100 for white in total and 6 out of 19 after
duplicate elimination. Patched: 43 wins in total and 5 out of 18 unique.
All the black wins are mirror-games until the end and thus black wins
them only by 0.5 due to the -0,5 komi. This leads to a mean score of 6.4
for white(!) out of the 19 games (not patched) and 5.9 out of the 18 games.
Don't know if this is stasticically significant.

> I'm not sure if there's any simple and safe strategy to break
> mirroring on 9x9. There are patterns to play around the edges and get
> first to capture, but there's a distinct risk that the opponent would
> deviate and come out better.

Unfortunately I don't think the 11 winning games in the attachment will
give any valuable suggestions - a human player could always deviate saving
his middle group and coming out better IMO.

After 50 19x19 games W: 3.5.10          vs.  B: 3.5.10 mirroring
and   50 19x19 games W: 3.5.10 patched  vs.  B: 3.5.10 mirroring
I got these results:
Not patched: 9 wins out of 50 for white in total and 9 out of 49 after
duplicate elimination. All the black wins against white not patched are
mirror-games until the end and thus black wins them only by 0.5 due to the
-0,5 komi. This leads to a mean score of 3.5 for white(!) out of the 49 games.

Patched: 41 wins in total and 36 out of 45 unique. Mean score is 20.4 for white.
Black get's through with mirroring only once - see: test-am_19_001-2.sgf .
This is due to W: M10 in move 6 which breaks gnugo's method.
(could that easily be avoided?)
But what concernes me more are games 5 and 18 that B won by 55.5 resp 38.5.
I'm not able to find crucial mistakes in them, so some review would be fine.

NB: Black wanted to resign several times after loosing tengen but unfortunately
    I had to turn that off to make the games count. (revising Gogui + tools now 
...)

Q: I had some difficulties applying the multi-file patch
   you supplied first. What is the proper way to apply it
   to a working copy with or without CVS?

happy GO - ing
     aquarius

P.S. will go on playing with the patch but please don't expect results as I will
     be offline for a week or so.

test-antimirror-white-9.tar.gz
Description: GNU Zip compressed data

test-antimirror-white-19.tar.gz
Description: GNU Zip compressed data