Re: [Bug-gnubg] Confidence intervals from rollouts

[Douglas Zare]

Quoting Joern Thyssen <address@hidden>:

> On Tue, Sep 03, 2002 at 12:53:21AM -0400, Douglas Zare wrote
> > There was an interesting question on the Gammonline bulletin board about
> the 
> > standard deviation in cubeless rollouts and the standard deviations of live
> 
> > cube results. I've included an excerpt below, in which I attempt to
> estimate a 
> > confidence interval for the difference between doubling and not doubling.
> I'm 
> > not sure what the best way to do this is, but I suggest that an attempt
> would 
> > be worth implementing in gnu.
> 
> I'm sorry but can you explain further :-)
> 
> A lot of the text below concerns the fact the Snowie doesn't calculate
> the standard error of the cubeful equity, however, gnubg does.

Snowie does provide a confidence interval for the live cube rollouts, but not 
the adjustments of cubeless rollouts. Anyway, the question is what is required 
to be able to conclude that doubling is better than not doubling (at least for 
your rollout parameters). You could wait until the confidence intervals for 
doubling and not doubling do not overlap. Even before then, if the rollouts 
were independent, you could stop when the two were separated by enough joint 
standard deviations. However, if you use a single cubeless rollout to estimate 
both numbers, then the numbers are highly correlated. That way you can have a 
higher confidence that doubling is correct, even if you don't yet know that not 
doubling has an equity less than A which is less than the equity from doubling.

Geometrically, after a rollout, you might have a density on the x-y plane of 
possible real equities after doubling (x) and not doubling (y). There might be 
an ellipse, analogous to the confidence interval, which is stretched in the 
direction of the y=x line, i.e., the projection to the x-axis is large, and the 
projection to the y-axis is large, but the projection to the line y+x=0 is 
small. You might know that y-x = 0.1 +- 0.03, while you only know that y=0.8+-
0.15 and x=0.7+-0.15.

By the way, I should have indicated more clearly that everything after the 
first paragraph in that excerpt was mine.

Douglas Zare