[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-gnubg] Confused

From: Timothy Y. Chow
Subject: Re: [Bug-gnubg] Confused
Date: Fri, 12 Jun 2015 12:20:32 -0400 (EDT)
User-agent: Alpine 2.11 (LFD 23 2013-08-11)

Lucas wrote:
Last year i tested using Fibs,( were i did in the past 8 bots), 2 bots
one set to play Worldclass and the other at grandmaster
so 2 ply against 3 ply
they played 3000 5 point matches
Worldclass the lesser setting had a winrate of 55 %

Ian Shaw wrote:
I'd be surprised if just 3000 5-point matches (maybe 12000 games) was sufficient to produce statistical significance.

A win rate of 55% for 3000 trials is significant at the 5 sigma level. Even if it turns out that the test was not statistically pure (an example of "impurity" would be failing to specify in advance the exact number of trials), Lucas's result is probably very significant.

Of course, what can be stated with high confidence is that the two settings are *not equally good*. One cannot state with equal confidence that 2-ply really does have a 55-45 advantage over 3-ply in 5-point matches.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]