bug-gnubg
[Top][All Lists]

## Re: [Bug-gnubg] opening book?

 From: Øystein Schønning-Johansen Subject: Re: [Bug-gnubg] opening book? Date: Fri, 29 Dec 2017 11:12:24 +0100

Thanks! I got this. I just didn't have time to reply.

I like some of this ideas. If an opening book can store the best move, maybe it can be extended to keep a cache of the best moves, maybe that can be even better. Chess engines does something similar, and calls this table "Killer moves". In the search the killer moves will be tried out first in hope to create a cutoff. On the other hand the effect of this may not be as great as in chess as there is no alpha-beta pruning in the search. In our case we just want to do the best move.

I have another suggestion:
when gnu plays money games it assumes both players have infinite amount of money but this may not be the case
an option to add how much money each player has and how much money corresponds to 1 point?
obviously if I have 10 millions for the game and you have 3 millions the bet can't get more than 3 million. If we agree that each point corresponds to 1 million then after a double the bet gets to 2 millions and then the next double is to 3 millions so not a true double. then the cube is dead.

Hmm... at one of the popular online sites (I don't remember the name of it) there was a concept of "Table stakes". I think someone looked into this, but I don't think it was ever implemented.

One last suggestion I have is contempt. Maybe gnu should play/double differently when playing against weak opponents?

This has been experimented with several times (I think Joseph did some experiments 10 years ago), but everything has failed, so far. I think  there is a missing factor in such cube decisions. I think if this scheme should be successful you need to add a "skill" estimator to position itself. Like some estimated number describing how much skill you need to play this position correctly. simple race -> low skill needed. Complex backgame -> high skill needed.  I have just some vague ideas on how to derive such number.

I have some more questions.
Can it handle the backgame well? I have heard that bots have trouble with backgame and in extreme situations may double in losing position. But is it still true for gnu now? Maybe strong players could beat it that way...
what is different in the new version? is it any stronger?

I don't think it is that bad. I think backgames are handled properly. However, it is one of those things it is hard to say. Should you compare to a human expert on backgames? How do you know the human expert is right?

There is a player called "udacity_capstone" * playing at FIBS. It only plays one-point-matches, so it does not care about gammons and backgammon. I see that it tends to go into backgames quite often. At least more often than I would expect. And I see that it plays it quite well. It doesn't do the "hit-too-early" mistake.

I do not have available time to implement any of these ideas now, so these ideas will be set up on the invisible backlog. Maybe someone else have the resources to look at this?

Best regards,
-Øystein

* "udacity_capstone" was a project I did for my udacity machine learning engineer nanodegree. It basically takes the training data of GNU Backgammon and retrains the main neural network for contact positions. It uses a deeper neural network structure. (I think it's 6 hidden layers now)

On Wed, Dec 27, 2017 at 10:43 PM, Aristidis Aristarxopoulos wrote:
Obviously, for different situations(score,if cube is used etc) the rollout would need to use those conditions.
I actually meant in money games like when you set match length to 0

1. What to store?
-the best move according to strongest rollout you could do for all the different situations, then play the move according to which situation it is. Or if this is too difficult then do it for money games only. are there different situations for money games? for example, does it matter if it is jacoby rule or not for the first moves?

2. should you store the cubeless probabilities after any opening move?
-I don't know whether cubeless or cubeful would make a difference for opening moves. I think that if the game is cubeless then the book should be using the best moves from cubeless rollouts and if the game is cubefull then the book should be using the best moves according to cubefull rollouts etc for different situations(although with match score it would be impossible to cover them all, maybe you could do for some situations like start of the game where the score is something used often like, 3, 7, 11 ,13... I assume that matches above 13 would have the same best moves but I am not an expert and I don't know for sure, am I mistaken? I also think that in a match in 7 points where one player is 6 away and the other 7 away, I think that the same moves would be the best as when the game starts, at least if the book is not extra deep.)
I don't know well how gnu works and whether storing the probabilities(cubeless or cubeful) would matter for how the program would play the next moves, do you mean to say to store the probabilities and then gnu uses them to make the moves? I meant that the book would have the moves so gnu would just make them(or maybe the gnu gui would make them). I think it would be nice to have the probabilities just so that one could see which they are out of curiosity using hint for the opening moves.

3. Or do you think it is wise to store the move (+roll) itself?
-That's what I was thinking, if gnu can make moves from the probabilities of a rollout then store the probabilities.if gnu can't then store the moves for each opening roll
I think you can have a book file that gnu reads so different books for different situations and then the user will load which one he wants... or maybe the user could make his own book choosing the moves for opening rolls
if making a book that can handle all situations is difficult then you can make separate books

4.Another approach could be to pre-fill the evaluation hash table with rolled out values for some early game position, that may be a good approach.
-I am not sure what this would do. Would it play the recommended moves from rollout? for all situations even?

I have another suggestion:
when gnu plays money games it assumes both players have infinite amount of money but this may not be the case
an option to add how much money each player has and how much money corresponds to 1 point?
obviously if I have 10 millions for the game and you have 3 millions the bet can't get more than 3 million. If we agree that each point corresponds to 1 million then after a double the bet gets to 2 millions and then the next double is to 3 millions so not a true double. then the cube is dead.

One last suggestion I have is contempt. Maybe gnu should play/double differently when playing against weak opponents?

I have some more questions.
Can it handle the backgame well? I have heard that bots have trouble with backgame and in extreme situations may double in losing position. But is it still true for gnu now? Maybe strong players could beat it that way...
what is different in the new version? is it any stronger?

Anyways, thanks a lot for the reply and sorry if I am getting you tired with this reply...
Please tell me if this text looks ok like the first I sent so if not I can send a separate email. Thanks again :)

Sent: Wednesday, December 27, 2017 at 3:46 PM
Subject: Re: [Bug-gnubg] opening book?
This is actually a good idea. I even think eXtreme Gammon uses some kind of opening book.

There are some challenges and practical details though.
What to store? should you store the cubeless probabilities after any opening move? Or do you think it is wise to store the move (+roll) itself? In case a book i storing the moves, the best move is actually depending on score, cube position, Crawford/pre-Crawford/post-Crawford etc. In that case we have to store the best move for each of the opening moves for each match score.... Hmmmm.... doable...sure, but is it feasible?

Another approach could be to pre-fill the evaluation hash table with rolled out values for some early game position, that may be a good approach.

-Øystein

On Tue, Dec 26, 2017 at 11:11 PM, Aristidis Aristarxopoulos wrote:
it seems that rollouts result in stronger moves.
Is it possible to make gnu play the opening moves and maybe the replies of them(etc) from an opening book containing best moves from rollouts?

_______________________________________________
Bug-gnubg mailing list