[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-gnubg] An evalutaion of the pruning nets
From: |
Robert-Jan Veldhuizen |
Subject: |
Re: [Bug-gnubg] An evalutaion of the pruning nets |
Date: |
Thu, 04 Nov 2004 16:04:20 +0100 |
User-agent: |
Mozilla Thunderbird 0.8 (Windows/20040913) |
Hi,
Some more about the pruning net results, I'll forward this from GOL (30
October):
***********************************************************************
I thought this was interesting:
Jim Segrave on the gnubg mailinglist, about the differences in a large
sample (although SE are still very high) of matches analysed by both
versions:
(...) The results:
..............................win.....wing...winbg...loseg...losebg cubeful
Average absolute difference 0.00123 0.00108 0.00009 0.00157 0.00020 0.00343
Std err.....................0.00619 0.00390 0.00051 0.00642 0.00403 0.02100
In 2,375 cases, the choice of best move differed, (0.94% of the time) (...)
[NOTE: Jim later changed this to 2.98% of the time]
I read some conclusions, probably based on this, that 2-ply prune is as
good as 2-ply no prune, practically speaking. I'm paraphrasing here,
sorry if I'm wrong about this.
However, looking at these figures I'm not so sure. In an absolute sense
these differences look very small indeed. However, 2-ply is supposed to
be playing a world-class (or better!) game. That means even the
slightest increase in error rates means a clearly lower level of play.
I'm not sure how to quantify and interpret the figures, certainly not
because the SE's are so high. But isn't it a bit early to draw
conclusions from here that the pruning has "almost zero" effect on skill?
From some simulations I did myself, mostly letting gnubg play against
itself, 0-ply vs. 2-ply and 2-ply reduced vs. 2-ply 100%, I think that
the pruning net choosing a different move almost 1% of the time [NOTE:
3% even, it seems] is significant, again considering the fact that 2-ply
is supposed to play world-class or better.
*************************************************************************
It seems like "different move" included all positions where moves have
equal equity so it doesn't matter. That makes the figures harder to
interpret; part of the 3% differences is simply irrelevant. But if
there's still 2% REAL differences, I think that's a significant
difference, also looking at the equity differences Jim reports here (but
which may not be of any value considering the SE?).
Greetings,
--
Robert-Jan Veldhuizen
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- Re: [Bug-gnubg] An evalutaion of the pruning nets,
Robert-Jan Veldhuizen <=