[Bug-gnubg] Is gnubg's game-analyisis function always reliable?

[Adi Kadmon]

Dear Developers

 

I have Version GNU Backgammon 0.90-mingw 20090403, and I’ve been training a lot with it and finding it very strong.

 

However, lately I’ve been encountering a phenomenon for which I cannot account. Analyzing the money-games (a single game each time) with some of my opponents using the feature “Analysis” (set to “supremo” 2-ply for checker play and to “grandmaster” 3-ply for cube decisions), I found consistently for some of the opponents that I was significantly better than they were by the analysis` measures, and nevertheless I consistently lost to them more than gain, sometimes in a ratio approaching 2:1 or 3:2!

 

For instance, Analyzing 69(!) games with a certain opponent, with the above settings for the analysis function, I found the following:

 

In 41 games I had an error rate mEMG per move (total: checker play + cube decisions) lower than his. The advantage was his only in 27, and in 1 game we came out equals.

 

In 43 games I had an error rate mEMG per move (checker play) lower than his, whereas he had the advantage in 26 games only.

 

In 29 games I had an error rate mEMG per cube decision lower than his, whereas he had the advantage in 21 only, and in 19 games we came out equals.

 

Seeing all this, it was all the more disconcerting to find that on average I lost 0.2826 points per game during these 69 games!

 

Note: the “marks” (“world-class…… awful”) for me and for him were consistent with the above analysis measures. The bot found my opponent to be luckier than me on average (unfortunately I didn’t keep the exact numbers of games on his side or mine) – but, for heaven’s sake, such luck through 69 game!!! With such an ostensible advantage on my side? It’s highly improbable. And this opponent is, as I said, not the only example.

 

I’ve been trying to imagine an explanation for these facts, such as:

 

(1)   That opponents, and a few more, happened to be definitely stronger than gnubg. But stronger than “supremo” and “grandmaster” in such a decided and consistent fashion – is it conceivable?

(2)   Those opponents somehow cheated heavily on the dice (on Play65 site) – but I deem it very unlikely, and I’m not used to being paranoid at all…

(3)   My opponents’ style was different from gnubg’s, to which I’m much more accustomed (and from which I indeed learned much), even though theoretically their moves are a bit inferior by its standards – and one has to “know how to win against their style” in order to succeed against them. But is such a thing likely? I would have rather thought that if by a very strong bot’s standards my moves are superior on average, it’s not a relativistic but a definite superiority and should beat a “different style”.

(4)   My errors occur in such early (or late) stages in the game in comparison with the opponent’s errors, that my errors are “more costly”, more decisive for the final result. However, I doubt this too: Wouldn’t the bot than have found my mistakes statistically very grave for the relevant positions, thereby affecting the above measures accordingly so that on average they again should turn out inferior, not superior?!

 

 

Well, I trust your vast knowledge and experience would yield a suitable explanation, or at least point to some possible explanations.

 

Please do help!

 

Sincerely,

 

Adi