Re: [gnubg] Help with a new MET (2)

[Joseph Heled]

Recent net improvements are not by me. (built on the the framework I
established, I think, but using modern computing power unheard of 15
years ago).

That is why I suggested a "weaker net". Unlike noise, it usually plays
weaker but not by doing silly things due to random noise. more like a
weaker human.

On Wed, 27 Nov 2019 at 14:04, Ian Dunstan <address@hidden> wrote:
>
> What do you think of using 0-ply with a little noise introduced?
>
> I have played around a little with that in the past and too much noise causes 
> Gnubg to do very silly things that I doubt a human would do.
>
> Another option might be to use a very old build of Gnubg. There are some 
> significant strength gains along the way to our current version. You did the 
> neural net, you would know that better than anyone :-)
>
> On Wednesday, 27 November 2019, 11:38:17 am AEDT, Joseph Heled 
> <address@hidden> wrote:
>
>
> On Wed, 27 Nov 2019 at 13:25, Ian Dunstan <address@hidden> wrote:
> >
> > Hi Joseph,
> >
> > "
> > That is why I, personally, would like to see proof it really matters. like:
> >
> >  -- Take a weak player - with the "weakest" MET file and the
> > "strongest" - and show a significant difference.
> >
> > "
> >
> > I understand your viewpoint. When Gergely and I can get the scripts and any 
> > bug-fixes sorted we will first do some preliminary testing as you suggest.
> >
> > I think 0-ply v 0-ply using the Woolsey MET vs the Kaz-XG2 MET might be the 
> > way to start.
> >
> > Then 2-ply v 2-ply using the Woolsey MET vs the Kaz-XG2 MET is probably 
> > needed as well for comparison.
> >
> > Let's see what comes from these sorts of tests and move on from there.
> >
> > I think I probably need to ask Xavier how he arrived at his supposedly 
> > 'significant difference' of 0.4 ELO between the ExtremGammon MET and the 
> > Roc-Kaz MET.
> >
>
> GNUBG ply-0 might not be weak enough :) I might be wrong here, but I
> "wanted" a weak player in the belief it will be easier to establish
> statistical significance. But I might be wrong and it is the other
> way. To get a weaker player you probably need to use a "weaker/lesser
> trained net".
>
>
> -Joseph
>
> > Cheers, Ian.
> > On Wednesday, 27 November 2019, 05:59:09 am AEDT, Joseph Heled 
> > <address@hidden> wrote:
> >
> >
> > Hi Ian and bg fans,
> >
> > I would like to clarify my own view RE your post and the other replies.
> >
> > Any BG playing bot is a complex entity. GNUBG uses two-sided bearoff
> > tables, one-sided bearoff tables, three different types of neural
> > nets, not to mention my "pruning nets" --  And all this just to guess
> > the cubeless probabilities. For match play it uses the MET file and
> > *an algorithm* to covert cubeless to cubeful. GNUBG uses the
> > code+formulas by Joern Thyssen, which I never understood fully, and
> > was always slightly skeptical of. My own cube code is based on linear
> > interpolating, based on the ideas here
> > (https://bkgm.com/articles/met.html) which I think is the right
> > approach (danalyze.cc in gnubg-nn).
> >
> > Yes, those are details, but they matter. The playing abilities of a
> > bot are a complex emergent property of the system. The MET table is
> > just one part, and focusing on it is <very-personal-opinion>enormously
> > premature</very-personal-opinion>. My own experience indicates the
> > system as a whole is not that sensitive to the MET. That is why I,
> > personally, would like to see proof it really matters. like:
> >
> >  -- Take a weak player - with the "weakest" MET file and the
> > "strongest" - and show a significant difference.
> >
> > Then we can start debating if it is worth while to "improve" the MET.
> > We have to use BOTS to develop this even if it is intended for human
> > use. Humans can't generate enough test data.
> >
> > I have more thoughts, but that is enough on this topic :)
> >
> > -Joseph
> >
> > On Tue, 26 Nov 2019 at 22:44, Ian Dunstan <address@hidden> wrote:
> > >
> > > Hi team,
> > >
> > > I have tried several subscription attempts to the email list directly, I 
> > > think, without success. After finding the archives I see that there have 
> > > been a number of replies to my first post. I didn't read them until very 
> > > recently I wasn't ignoring your feedback, I just had not viewed it. Thank 
> > > you, all, for what I received. Also, I apologise that this post almost 
> > > certainly starts a new thread, though I considered it more important to 
> > > make a response now.
> > >