Re: [Bug-gnubg] Rating and grading

[Joseph Heled]

Rod Roark wrote:
> On Monday 07 July 2003 10:40 am, Albert Silver wrote:
>
> > ... GNU's attributed rating is supposedly based on the FIBS rating system
> > and is designed to give a rating that approximates what a user might get
> > at FIBS. However, it's self-attributed rating seems to be what
> > everything else is relative to and is at 2200. GNU does not have a 2200
> > rating at FIBS, not even by a margin. I think that lowering this to
> > 2050, representing it's average, would make much more sense. As a
> > consequence, all its relative ratings should be lowered accordingly.
>
>
> Here are the latest cumulative stats for ParlorBot, which
> runs gnubg on FIBS with 2-ply checker play and cube
> decisions, 33% reduced:

I have found that 33% reduced for checker play is not really good. Using 
small pruning nets give you the speedup with a much reduced error rate.

> Number of samples  : 3692   (i.e. number of rated matches played)
> Lowest             : 1880.71
> Highest            : 2120.74
> Mean               : 2002.35
> Standard Deviation :   51.70
>
> While the average rating used to be around 2030, there has
> been a great deal of wandering from that.  Also I suspect
> that a few clever FIBSters have learned to take advantage of
> the bot's less-than-perfect evaluation in back games and
> holding games.

The FIBS rating formula is not perfect. In my small tests (using 2 
different nets as players) the weaker player has an advantage, i.e. wins 
more of his 3,7,9,... matches than what is implied by the 1ptr win rate. 
So, if you stuck mainly playing against weak players, your rating drops. 
If you play stronger ones, it goes up.

We need lots of data - i.e. matches between GNUbg and players - analyzed 
their error rate - and try to come up with a formula that better matches 
the FIBS number.

-Joseph

> -- Rod
>
>
>
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/bug-gnubg