This post doesn’t represent that of any other developer besides myself, and doesn’t necessarily reflect the views of the others.
---------
There are things that you have said that are correct and others you draw conclusions on based on no evidence provided here. In some cases I will ask questions and then follow up based on your responses
“The interesting thing to note is that GNUBG also cheats.”
Okay, so you are going to prove that to us in your post. So lets not comment on this yet. However, to declare the bot a cheater you would have to prove that the bot knows something that the human can not POSSIBLY know to make a decision.
You said: “The engine is scrupulously honest and there is NO hidden trapdoor in the code.”
I give you this point. Agreed.
You said:“What I am denoting is that the NNP "learns" the attributes of the specific roll generator you use”
This is true to a degree. If there is a bias in the PRNG with enough training the bot could inherently indirectly learn based on taking advantage of the bias. I mean lets say I create an PRNG that deliberately rolls a bias towards double 3, and train it with a billion (some arbitrarily high number) it does reason that the weights file could be skewed based on the biased view of randomness that was fed into it. So lets say for the sake of argument you are correct, and that Gnubg learned backgammon based on skewed dice and ultimately had a weighting file generated out of that bias.
You said:“And does it probably better than a human can.”
Are you telling me that you know someone who can perceive the difference (and statistically validate the claim) without the aid of software? I can tell you now “probably” is an understatement. In fact I think you may overestimate the humans ability to recognize a pattern (by themselves) in most modern day random number generators (An there are a number of them now). It should read “And does it better than a human can”.
You said:“It will also learn the average hidden "tell" in the pattern of a human player; and the full DB in all likelihood (I so assert) has several folded state transition spaces covering the panoply of the subtypes of human play available. (given enough games rolled in).”
This is true. The neural net wasn’t entirely trained by the bot playing against itself. World class players also manually trained it. Inherently GnuBG has been given to some degree knowledge of world class players (and their playing style). Had the best “Backgame” player in the world trained enough “back game” positions into the bot, the neural net would be better in that regard. But based on all available data (both self learning and from world class players) the bot is biased towards believing that backgammon is best played as a “race”. You could call the integration of World Class player training and subsequent absorption into the neural net to allow a bot to get a “tell” on the way the human may potentially play.
So I don’t dispute this either. But here is something I will say. It is not cheating to “exploit” the knowledge that it has learned about how humans play. I bully weaker opps with cubes because I know some people have tendencies to play them badly. I capitalize on that to increase my chances to win. That’s not cheating, that’s just playing to the strength and weaknesses of the opponent. So on this point, if the bot has an inherent tell on human play, it doesn’t make it a cheater. So I move on.
You said:“You can tell alot about each player by their wn pattern;”
“and a true expert will be able to infer more about one player by how they react to another than you might believe.
Agreed. See above. If the bot was never trained with manual training (from humans), then the bot will not have any tells on humans really play/perceive the game. Knowing your opps playing patterns is not cheating.
You said: “Basically, _everything_ goes into it. “
This is way too general. What goes into it is any knowledge that can be directly measured and learned from. The direct inputs can be used, and bot alter its strategy based on additions/changes in the NN weights. It is possible for the bots to inherently garner knowledge and discover patterns that we may not even think about.
You said: “And whether you fold the state space down from the infinite manifold caleed reality
into a mere 100,000 trained transition nodes”
Yes, but those limited number of trained nodes have to represent all of the data from the known universe (The data fed to the bot). Compared to the amount of data actually processed a lot of it has been thrown away as noise, in coming up with a final engine, and weights file. However some patterns may inherently learned and become part of those weights. That recognition is not in itself cheating (I know where you are going with this, I will defer commentary until later on)
You said: “_everything_ goes into it.
And _everything_ in it that -can_ come out __will__”
Not necessarily. Even if there are patterns the bot may not have measured them as anything that affected the outcome and weights. Some will of course. Not all data that goes in necessarily was evaluated to be useful by the bot. We technically don’t know what hidden (and implied) knowledge is lurking in the weights file as seen by the NN.
I snipped a section out because it doesn’t add much to what has been said. We start getting into the heart of your argument here though:
You said: “then take it on faith that the program involved has found aparticular _weakness_ inherent in the play of either the human game platers loading the NNP state space or a hidden pattern in the PRNG series being used.” and previously you said “You said: “The engine is scrupulously honest and there is NO hidden trapdoor in the code.”
So lets say hypothetically you are right for argument sake. That Gnubg happened to stumble upon its limited weights file implicit understanding of some perceived bias (patterns) in the random number stream. Lets just say IT IS the case (I can’t prove it isn’t), but what I can say is this. There is no backdoor as you stated earlier, and the bots NN isn’t unscrupulous. I can tell you that no inputs into the NN (as you probably are already aware) include the source of the randomness. So all knowledge of any bias is inside the NN/Weights.
So given this above, and the assumption you are right. Care to tell us which random number generator(s) were used in the Training of Gnubg (I do know which one was used in the original 0.0 and 0.1 neural net trainers – the source Is available) . But here is food for thought. If I create a new Random number generator (lets say both a new real world RNG and a PRNG) and add support into GnuBG and don’t change the neural net or the weights – can the bot possibly know patterns for random number generators it hasn’t seen. Well I say “No”. The information used during training did not include bias or skew for random number generation (real world or PRNG) across all of space and time.
If someone believes there is a bias inside the neural net regarding randomness, then you should tell us which generators helped generate the bias because the simplest solution is to use a random number source that is different.
“Even random.org/com <http://random.org/com> notes that their random series are "improved" (made "more" effectively or "intuitively" random) by removing or reducing large series of improbable rolls (such as 0's and 1's in a row).”
Actually, you are wrong. It doesn’t arbitrarily throw away ones and zeros (it does but your view is too simplistic as stated here to give an accurate representation of what they really do). They do remove bits that are repeated but the algorithm does so for both 1’s and 0’s. It Is not biased in that regard. You can’t get less random data by applying a filter (like a noise filter,( to the existing data, as long as the filter applies the “SAME” logic to both 0 and 1 uniformly. On top of this something that also occurs is that data retrieved from the physical data source may not contain 100% entropy. So to begin with a lot of data retrieved from the real world is literally tossed away, and only the parts with the most entropy are used”.
As an example of this lets say I have a physical data source that is random. It just so happens that 3 bits of data are always 1. If I know the lower 5 bits of data contain the entropy I simply throw away the top 3 bits. Throwing away data that doesn’t add anything to the randomness is valid. The guys at random.org over the years take the data from multiple sources and then XOR the data together (I think they use sounds cards still)
In other cases random numbers are generated from a pile of 1’s and 0’s (from physical sources) , but the actual ones and zeroes of the raw data are not random at all. This is the case if you have a fixed sample rate and the source of the randomness is in the time between successive transitions from 0 to 1, not the actual transitions themselves. So basically the randomness is temporal in nature. You effectively go down the data stream and look how long an event of 0’s and 1’s last and the time they appear is used to decide . This throws away a huge amount of data since the transitions are far less frequent than the total amount of data.
Throwing away data uniformally with no bias doesn’t mean the data is less random, it just simply means you are looking for the bits with entropy and the bits that aren’t noise.
“It is really difficult to predict what is actually in it, and whether it can "beat" the :"cheat" that the GNUBG database gives the engine.”
“Its difficult to predict whats actually in it”. So I will ask, did you do a statistical study across a huge sample of matches to prove that there is an inherent difference between the results of matches done by random number generators known to Gnubg and a source of randomness that is unknown. Remember, you said ““The interesting thing to note is that GNUBG also cheats.”. That’s a definitive statement. You didn’t say ““The interesting thing to note is that GNUBG MIGHT cheats.””. If the bot has knowledge of how a player played (hypothetically) and used it to gain an advantage – that is not cheating. The human player can do the same thing when playing the bot – take advantage of the bots weakness and exploit them. Same goes for 2 human players. If one player has no knowledge of his opp and has no idea about any potential “tells”, but the opponent has knowledge of the opponent and his tells – that doesn’t make the more knowledgeable player a cheater – it makes him more informed. If one player knows how to compute an accurate pipcount in his head and the opp can’t, that doesn’t make the player who can do it a cheater. If a player knows about match equity tables and the MWC at a particular score, and the opp has no idea what a MET is... It doesn’t make the player who knows – a cheater.
IF a human is told a data source (Lets say a given PRNG) is random, and lets just say hypothetically Gnubg knew that wasn’t the case implied via the inputs, and the weight files – could that be cheating? Yes, one could say it is, the bot has been given a piece of information that needs to be equally shared by both players to be fair (What the bot and human do with it is up to them). But the thing here is, no where in your post have you said how the neural net got its random data (which algorithm(s), and whether the match being played was using the same one. And if you feel that Gnubg has an unfair edge, then Gnubg developers have given you 2 methods - “External Dice” and “Manual Dice”.
It sort of interesting. Lets say the NN was done with manually rolled dice (Hypothetically) and precision dice weren’t used and thrre was a known bias toward 5. Lets say Gnugb learned from all of that data, and starte dto play Backgammon, but its evaluation of a position and its value is based partially on those 5’s showing up more often (and the 2 showing up less often). Gnubg would learn to play BG in that environment but it could work against the bot if someone replaced the bad dice used during training with good precision dice used during matches played against the bot later on. Its possible that the implicit learned bias would work against the bot when different dice were used.
I have seen nothing provided that proves Gnubg cheats. There is no evidence or data supplied (If you have some I’d be happy to look at it) that there is any knowledge by the bot. You state ““It is really difficult to predict what is actually in it, “. So you must have done the difficult task to prove that patterns in the randomness do make a difference, and I wish to see that data. Until that’s done I don’t buy your first statement “The interesting thing to note is that GNUBG also cheats.”. And if you want to believe the bot cheats because it has an inherent accumulated knowledge of patterns of randomness in how it evaluates the equity in a position then I say use a different generator! The Code is open source, we’d be glad to see you add a new generator to the mix that you know wasn’t used during training.
My opinion Is that you have given a flimsy argument to say the bot cheats because of some knowledge about PRNG’s that it learned indirectly during its training with no evidence to support the position. I am going to say that knowledge is power. GnuBG knowledge doesn’t make it a cheater. It has a match equity table built in. That doesn’t make it a cheater, it makes it more informed. The bot can evaluation 100s of thousands of possibilities very quickly, humans can’t. Does that make the bot a cheater? No. It makes it knowledgeable. I am beginning to think (And I suggested this with Murat) that I’d be curious to see how one is defining “cheater”.