[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-gnubg] Alternative weights files and call for benchmarkers (lon
|
From: |
Philippe Michel |
|
Subject: |
Re: [Bug-gnubg] Alternative weights files and call for benchmarkers (long) |
|
Date: |
Mon, 25 Jun 2012 21:36:03 +0200 (CEST) |
|
User-agent: |
Alpine 2.00 (BSF 1167 2008-08-23) |
On Mon, 25 Jun 2012, Joseph Heled wrote:
Hi again Philippe,
Did you find a way to show that the new net indeed is indeed more balanced
than the old with regard to the odd-even ply syndrome?
I can't prove it or show statistically credible evidence. What's clear is
that 1ply is much better than before (and presumably so is 3ply, to some
extent ; it would take perr.py a few days to check that).
For this point, I'm certain it comes mostly from having both sides of each
position.
Race benchmarks went from :
0p errors 16847 of 96620 cost 56.8900107385 avg 0.000588801601517
0p errors > 0.08 2 of 96620 cost 0.327563
0p errors > 0.02 309 of 96620 cost 9.25185833665
0p errors > 0.005 3379 of 96620 cost 37.2425312228
n-out ( 488 ) 0.51%
1p errors 18637 of 96620 cost 85.8714767845 avg 0.00088875467589
1p errors > 0.08 22 of 96620 cost 3.042795
1p errors > 0.02 855 of 96620 cost 28.503381908
1p errors > 0.005 4668 of 96620 cost 64.9310108204
n-out ( 1157 ) 1.20%
2p errors 11707 of 96620 cost 30.0686683717 avg 0.000311205427155
2p errors > 0.08 1 of 96620 cost 0.227432
2p errors > 0.02 81 of 96620 cost 2.40806460092
2p errors > 0.005 1707 of 96620 cost 16.5559574447
n-out ( 384 ) 0.40%
to :
0p errors 17247 of 96620 cost 54.0759081675 avg 0.000559676135039
0p errors > 0.08 0 of 96620 cost 0.0
0p errors > 0.02 277 of 96620 cost 7.63623908773
0p errors > 0.005 3308 of 96620 cost 34.7364841505
n-out ( 1017 ) 1.05%
1p errors 15765 of 96620 cost 58.601870743 avg 0.000606519051367
1p errors > 0.08 7 of 96620 cost 0.635476
1p errors > 0.02 473 of 96620 cost 14.9148278984
1p errors > 0.005 3257 of 96620 cost 40.5046737865
n-out ( 944 ) 0.98%
2p errors 12000 of 96620 cost 30.2239266595 avg 0.000312812323116
2p errors > 0.08 0 of 96620 cost 0.0
2p errors > 0.02 100 of 96620 cost 2.78799052643
2p errors > 0.005 1737 of 96620 cost 16.9867535779
n-out ( 640 ) 0.66%
which looked good (the 2 ply numbers are puzzling, maybe fixing the n-out
positions would clarify that).
For the crashed network, it was elating. From :
0p errors 24169 of 99922 cost 598.866351999 avg 0.00599333832388
0p errors > 0.08 1183 of 99922 cost 163.331272
0p errors > 0.02 9412 of 99922 cost 477.0309225
0p errors > 0.005 18948 of 99922 cost 586.30953419
n-out ( 578 ) 0.58%
1p errors 24422 of 99922 cost 624.1492517 avg 0.00624636468145
1p errors > 0.08 1256 of 99922 cost 170.2153864
1p errors > 0.02 9953 of 99922 cost 503.9643812
1p errors > 0.005 19344 of 99922 cost 612.20677169
n-out ( 965 ) 0.97%
2p errors 18470 of 99922 cost 348.172686826 avg 0.00348444473515
2p errors > 0.08 479 of 99922 cost 77.0957015
2p errors > 0.02 5240 of 99922 cost 247.6708315
2p errors > 0.005 13183 of 99922 cost 335.8279518
n-out ( 292 ) 0.29%
to :
0p errors 24072 of 99922 cost 555.52707726 avg 0.00555960726627
0p errors > 0.08 922 of 99922 cost 118.7465472
0p errors > 0.02 9172 of 99922 cost 431.8378972
0p errors > 0.005 18854 of 99922 cost 543.04263622
n-out ( 0 ) 0.00%
1p errors 21602 of 99922 cost 428.84248938 avg 0.00429177247633
1p errors > 0.08 530 of 99922 cost 71.2716067
1p errors > 0.02 7137 of 99922 cost 312.1260868
1p errors > 0.005 16292 of 99922 cost 416.40087626
n-out ( 271 ) 0.27%
2p errors 17940 of 99922 cost 302.613062433 avg 0.00302849284875
2p errors > 0.08 299 of 99922 cost 47.4546694
2p errors > 0.02 4757 of 99922 cost 203.5541814
2p errors > 0.005 12586 of 99922 cost 290.29385724
n-out ( 18 ) 0.02%
For the odd/even effet itself, I have only anecdotical evidence, things
like what follows (old net first, then new one). I don't really know how
much it improved and the relative influence of better equities in the
training database and the presence of both sides of each position, but I'm
confident it is attenuated in most cases.
Position ID: /94AABCz3QwgBA
Match ID: QgkLAHABMAAA
Evaluator: Crashed
Win W(g) W(bg) L(g) L(bg) Equity Cubeful
static: 0.782 0.047 0.000 0.033 0.000 +0.579 +0.505
1 ply: 0.809 0.054 0.000 0.032 0.000 +0.641 +0.576
2 ply: 0.800 0.042 0.000 0.042 0.000 +0.601 +0.535
3 ply: 0.814 0.045 0.000 0.035 0.000 +0.638 +0.578
4 ply: 0.813 0.037 0.000 0.040 0.000 +0.624 +0.566
Win W(g) W(bg) L(g) L(bg) Equity Cubeful
static: 0.812 0.042 0.000 0.025 0.000 +0.640 +0.576
1 ply: 0.814 0.043 0.001 0.035 0.000 +0.637 +0.573
2 ply: 0.814 0.038 0.000 0.035 0.000 +0.631 +0.570
3 ply: 0.822 0.036 0.001 0.035 0.000 +0.646 +0.588
4 ply: 0.823 0.035 0.000 0.036 0.000 +0.645 +0.590
Position ID: //EAARDs7gYECA
Match ID: QQkFAMAAAAAA
Evaluator: Crashed
Win W(g) W(bg) L(g) L(bg) Equity Cubeful
static: 0.920 0.337 0.000 0.011 0.000 +1.166 +1.138
1 ply: 0.944 0.382 0.008 0.007 0.000 +1.270 +1.251
2 ply: 0.932 0.330 0.000 0.010 0.000 +1.184 +1.161
3 ply: 0.941 0.370 0.006 0.009 0.000 +1.250 +1.230
4 ply: 0.935 0.325 0.000 0.010 0.000 +1.184 +1.162
Win W(g) W(bg) L(g) L(bg) Equity Cubeful
static: 0.935 0.411 0.002 0.007 0.000 +1.276 +1.254
1 ply: 0.949 0.387 0.011 0.006 0.000 +1.290 +1.273
2 ply: 0.939 0.374 0.005 0.009 0.000 +1.248 +1.227
3 ply: 0.945 0.372 0.010 0.008 0.000 +1.264 +1.245
4 ply: 0.939 0.363 0.006 0.010 0.000 +1.238 +1.217
Position ID: dwwA8AeVbRsECA
Match ID: QYkGAAAAAAAA
Evaluator: Contact
Win W(g) W(bg) L(g) L(bg) Equity Cubeful
static: 0.945 0.579 0.010 0.006 0.000 +1.473 +1.454
1 ply: 0.966 0.680 0.019 0.004 0.000 +1.626 +1.614
2 ply: 0.949 0.579 0.008 0.004 0.000 +1.480 +1.463
3 ply: 0.964 0.651 0.020 0.003 0.000 +1.596 +1.584
4 ply: 0.950 0.573 0.007 0.003 0.000 +1.477 +1.460
Win W(g) W(bg) L(g) L(bg) Equity Cubeful
static: 0.947 0.600 0.008 0.006 0.000 +1.494 +1.476
1 ply: 0.959 0.642 0.026 0.005 0.000 +1.580 +1.566
2 ply: 0.953 0.605 0.008 0.004 0.000 +1.516 +1.500
3 ply: 0.957 0.624 0.029 0.004 0.000 +1.564 +1.549
4 ply: 0.953 0.603 0.007 0.004 0.000 +1.514 +1.499