Re: [Bug-gnubg] Confused

[Timothy Y. Chow]

Lucas wrote:
> Last year i tested using Fibs,( were i did in the past 8 bots), 2 bots
> one set to play Worldclass and the other at grandmaster
> so 2 ply against 3 ply
> they played 3000 5 point matches
> Worldclass the lesser setting had a winrate of 55 %

Ian Shaw wrote:
> I'd be surprised if just 3000 5-point matches (maybe 12000 games) was 
> sufficient to produce statistical significance.

A win rate of 55% for 3000 trials is significant at the 5 sigma level. 
Even if it turns out that the test was not statistically pure (an example 
of "impurity" would be failing to specify in advance the exact number of 
trials), Lucas's result is probably very significant.

Of course, what can be stated with high confidence is that the two 
settings are *not equally good*.  One cannot state with equal confidence 
that 2-ply really does have a 55-45 advantage over 3-ply in 5-point 
matches.

Tim