RE: [Bug-gnubg] Snowie error rates versus gnubg error rates

[Albert Silver]

Just one comment: another fundamental difference between the way the two
programs calculate ratings is in the inclusion or not of unforced moves.
Snowie definitely includes them, which would also ease the way grades
come out. I had a discussion on this at the forum at the Snowie site and
was told that if they didn't, we'd all be judged as beginners all the
time. I'm not going to enter this discussion as I think NOT including
unforced moves far more intelligent, however, if this difference isn't
somehow taken into account then GNU is not going to be the same as
Snowie, it's going to be a LOT harder. I mention this because it just
seems to me, from empirical experience alone, that 0.0083 as the bottom
limit of Expert and 0.012 as the bottom limit of Advanced seems a little
strict. I know Snowie has in practice been stricter in its grading, but
I didn't get the impression it was THAT strict. 

                                                Albert

> -----Original Message-----
> From: address@hidden [mailto:bug-gnubg-
> address@hidden On Behalf Of Joern Thyssen
> Sent: Sunday, April 06, 2003 4:41 PM
> To: GNU Backgammon Bugs
> Subject: Re: [Bug-gnubg] Snowie error rates versus gnubg error rates
> 
> On Sun, Apr 06, 2003 at 12:10:03PM +0000, Joern Thyssen wrote
> 
> > ad (a): I've recieved a sample of 50 matches from Albert. I can use
them
> > to deduce an empirical conversion factor between gnubg and snowie
error
> > rates. This can be used to adjust the threshold for assigning the
> > ratings "supernatural", "world class" etc, so gnubg and snowie are
more
> > aligned.
> 
> I've analysed close to 300 matches (50 from Albert and 250 from my own
> archive), and the average conversion factor is 1.38 (std.dev. 0.37).
> 
> Snowie's thresholds are:
> 
> 0.0 - 1.2: Extra-terrestrial
> 1.2 - 4.4: World class
> 4.4-5.9: Expert
> 5.9-8.8: Advanced
> 8.8-12.6: Intermediate
> 12.6-18.5: Beginner
> 18.5 - ??: Novice
> 
> I suggest the following change (pending the discussion):
> 
>                 gnubg's old               gnubg's new
>                                           (based on a factor of ~1.4)
> Supernatural    0.000 - 0.005             0.0000 - 0.0017
> World Class     0.005 - 0.010             0.0017 - 0.0062
> Expert          0.010 - 0.015             0.0062 - 0.0083
> Advanced        0.015 - 0.020             0.0083 - 0.012
> Intermediate    0.020 - 0.025             0.012  - 0.018
> Casual Player   0.025 - 0.030             0.018  - 0.026
> Beginner        0.030 - 0.060             0.026  - 0.035
> Awful!          0.060 - oo                0.035  - oo
> 
> Jørn
> 
> 
> _______________________________________________
> Bug-gnubg mailing list
> address@hidden
> http://mail.gnu.org/mailman/listinfo/bug-gnubg