Re: [Bug-gnubg] Measuring performance levels

[Douglas Zare]

Quoting Joseph Heled <address@hidden>:

> Morten Wang wrote:
> > 
> > * Joern Thyssen
> > > Yes, uncheck cubeful rollout and uncheck cubeful chequerplay. Turn off
> > > the Jacoby rule.
> > 
> > thanks to both of you!
> > 
> > I figured both those checkboxes for cube usage would have to be
> > unchecked, but I'd forgotten the Jacoby rule.

Oddly, Snowie 3 includes as an additional statistic the DMP equity, but I don't 
think anyone in the world uses that value. (I may be wrong. I know someone who 
used to check the Thorp count to decide whether to double with a 5-prime until 
I stopped him.) The standard money cubeless equity is much more useful, and I 
use that frequently.

> > I did the 21600 0ply rollout for reference, and here's what I got:
> > 
> > Cube analysis
> > Rollout cubeless equity  +0,516
> > 1. Double, take         +0,619
> > 2. Double, pass         +1,000  ( +0,381)
> > 3. No double            +0,335  ( -0,284)
> > Proper cube action: Double, take
> > 
> > Rollout details:
> > Centered 1-cube:
> >   0,758 0,069 0,000 - 0,242 0,068 0,000 CL  +0,504 CF  +0,335
> >  [0,050 0,024 0,028 - 0,050 0,012 0,000 CL   0,113 CF  +0,138]
> > Player O owns 2-cube:
> >   0,755 0,069 0,000 - 0,245 0,068 0,000 CL  +0,998 CF  +0,619
> >  [0,049 0,024 0,028 - 0,049 0,012 0,000 CL   0,223 CF  +0,282]
> > Full cubeful rollout with var.redn., with rotate rolls
> > 21600 games, seed 11 using Mersenne Twister
> > Play and cube: 0-ply cubeless
>  
> I think Doug insisted on Cubeless. All my rollouts show around a 3.7 gammon
> rate, while (if I read this correctly) this rollout gives 6.9
> 
> So, is this really a cubeless rollout? What else can be wrong?

I'd like this to be a cubeless rollout because otherwise the side being 
contained can make mistakes, and I want that side to play essentially 
perfectly. 

I did a 999,999 trial rollout with Snowie 3 1-ply, and it indeed indicates that 
the first rollout was lucky.

Seed 99:
Cubeless equity 0.489 +- 0.002 (confidence interval).
0.1 3.4 76.5 -- 23.5 7.5 0.0

Since the standard deviation for the cubeless equity is about 0.001, the 
standard deviation for the win rate should be about 0.0005, or 0.05%. Are the 
second lines, e.g., "[0,050 0,024 0,028 ..." supposed to represent standard 
errors for the gnu rollout? Those seem much too high even with no variance 
reduction, unless it is 0.05%, 0.024%, etc. 

Also, with those figures, I think the cubeful equities should be much closer to 
1. It's not a trivial take.

Douglas Zare