
From:  Timothy Y. Chow 
Subject:  Re: [Buggnubg] Confused 
Date:  Fri, 12 Jun 2015 12:20:32 0400 (EDT) 
Useragent:  Alpine 2.11 (LFD 23 20130811) 
Lucas wrote:
Last year i tested using Fibs,( were i did in the past 8 bots), 2 bots one set to play Worldclass and the other at grandmaster so 2 ply against 3 ply they played 3000 5 point matches Worldclass the lesser setting had a winrate of 55 %
Ian Shaw wrote:
I'd be surprised if just 3000 5point matches (maybe 12000 games) was sufficient to produce statistical significance.
A win rate of 55% for 3000 trials is significant at the 5 sigma level. Even if it turns out that the test was not statistically pure (an example of "impurity" would be failing to specify in advance the exact number of trials), Lucas's result is probably very significant.
Of course, what can be stated with high confidence is that the two settings are *not equally good*. One cannot state with equal confidence that 2ply really does have a 5545 advantage over 3ply in 5point matches.
Tim
[Prev in Thread]  Current Thread  [Next in Thread] 