Re: [Bug-gnubg] Gnubg pubeval score

[Ian Shaw]

It looks like gammons were counted, but not backgammons, as is traditional in 
the Greek version. “The
bot is awarded a +1 point for a single win, +2 points for a double win, -1 for 
a single loss, -2 for a double loss.”

https://lists.gnu.org/archive/html/bug-gnubg/2012-01/msg00034.html is the 
original source for gnubg vs puveval = +0.6.

> Playing a 1 point match at 0 ply, gnubg won 7193 of 10000 matches,
> which is 0.4386 ppg in my book.
> Playing 10000 money sessions (gammons count, no cube) gnubg got 0.6296
> ppg (the breakdown is 4504, 2298, 285   2202, 676, 35 )
>
> -Joseph

Joseph reports the equity as 0.6296, but he counted backgammons as ±3 points. 
Counting backgammons as ±2 points gives the value of 0.6046 cited in 
Papahristou & Refanidis.

This is from 2012. Have there been improvements to the nn since then? I 
remember being able to "set player 1 pubeval", but this option no longer seems 
to be available. Else I'd repeat the test. 

* Ian 

From: Bug-gnubg [mailto:address@hidden On Behalf Of Øystein Schønning-Johansen
Sent: 23 January 2019 06:42
To: Robert Edgar <address@hidden>
Cc: address@hidden
Subject: Re: [Bug-gnubg] Gnubg pubeval score

How does the paper report this? Does it take gammon and backgammons into 
account (ie. cubeless moneygame) or does it just count win/loss ratio (ie. like 
one-point-match) ?

-Øystein

On Tue, Jan 22, 2019 at 11:05 PM Robert Edgar <address@hidden> wrote:
Can anyone confirm the score of a recent version of gnubg vs. pubeval? I 
hacked the source and found that gnubg v1.06 averaged +1.1ppg (82% wins) 
over 10k games, but a recent paper Papahristou & Refanidis (2017) quotes 
+0.60 ppg which is only marginally better than TD-Gammon (+0.59). My 
number seems high, but +0.6 seems too low considering how much effort 
went into optimizing the gnubg code.


_______________________________________________
Bug-gnubg mailing list
address@hidden
https://lists.gnu.org/mailman/listinfo/bug-gnubg