Re: [Bug-gnubg] FW: gnu bug on vista?

[保坂範行]

Very simple test would be :
> echo 'quit' | ./gnubg -b -r -t -q

another one.
> echo 'new match 3\nresign 3\nanalyse match' | ./gnubg -t -q

These are smoke test rathter than unittest.

Personally, I do
>./gnubg -t -p test.py
with python script following:
import gnubg
gnubg.command('import mat "test.mat"')
gnubg.command('analyse match')
gnubg.command('save match "test.sgf"')
print 'done'

Of cause, you need test.mat.
To evolute this to a regression test, need to put reference.sgf in
repo, and diff them.

Is there listed or archived gnubg errorious positions in past?
And I need "reference" revision of gnubg... othere wise I will never
know which is 'correct".
If any, I want to turn it into automated regression test suite with
python script.


I made small patch to give Python interpreter in gnubg sys.argv.(see attachment)
what does it  mean:
> ./gnubg -t -p sysargv.py this is argv
with python script sysargv.py following:
import sys
assert sys.argv[1] == 'this'
assert sys.argv[2] == 'is'
assert sys.argv[3] ==' argv'

I think it may help testing a lot, by parameterizing script with
datafile for testing.



Another idea, but may needs lots of work is "recoding/playing UI event to file".
Big rewards for this are
- easing end user level report of bug... gnubg it self record UI
events. end user just sends a record file.
 with great bug reproduction odds.
- regression test/unittest for UI.

One extension of this idea would be having scripting API for this UI
event serialization.
ie. injecting UI event via script language.


Nori


2009/3/10 Øystein Johansen (OJOHANS) <address@hidden>:
> Hmmmm....
>
> Just how I see things: a speed test is something for the end user. It does 
> not do anything useful except indicating the speed of the evaluations on the 
> preent computer. The end user can not toggle any setting anyway so she can 
> only use the result for bragging and comparing her different computers. 
> Nori's suggestion of a hardcoded seed is ok.
>
> Developers and builders should have other tests. A suit of tests that can be 
> run automatically to check functionality (and maybe speed). A kind of unit 
> tests. This set of tests could be in the cvs repository. The Gtest 
> environment in glib should be usable for such feature.
>
> -Øystein
>
>>-----Original Message-----
>>From: address@hidden
>>[mailto:address@hidden On
>>Behalf Of Christian Anthon
>>Sent: Tuesday, March 10, 2009 1:58 PM
>>To: 保坂範行
>>Cc: address@hidden
>>Subject: Re: [Bug-gnubg] FW: gnu bug on vista?
>>
>>Yes we should test at least play, hint, analysis and rollout.
>>
>>Christian.
>>
>>On Mon, Mar 9, 2009 at 1:19 PM, 保坂範行 <address@hidden> wrote:
>>>> Having something close to a non-regression test would be just great.
>>>> First idea would be to have it embedded into gnubg's code,
>>maybe as a
>>>> python script.
>>>
>>> I have a naive quesiton.
>>>
>>> Why not let it play gnubg v.s gnubg with a fixed initial seed?
>>> 1) it should reproduce becuase it has fixed initial seed.
>>> 2) seed will be very small to carry around.
>>> 3) we can expect typical occurence of positions as usual
>>daily backgammon games.
>>> 4) Once captured the evaluation of that match, we can verify later
>>> build with that capture.
>>>
>>> Nori
>>>
>>>
>>> On Mon, Mar 9, 2009 at 9:02 PM, Massimiliano Maini
>>> <address@hidden> wrote:
>>>>
>>>> address@hidden wrote on
>>>> bug-gnubg-bounces+09/03/2009
>>>> 12:55:15:
>>>>
>>>>> Øystein Johansen (OJOHANS) wrote:
>>>>> >> Perhaps we should just disable the mt speed test, until
>>somebody
>>>>> >> fixes it.
>>>>> >>
>>>>> >
>>>>> > Yes! Remove it until further. The speed test should be reworked
>>>>> completely. Today it generates a random position and then
>>evaluates
>>>>> the position -- and repeats this. However, picking a
>>random position
>>>>> is quite strange since you seldom get anything else than contact
>>>>> positions (Take a look at the generated positions, I dare
>>you!). The
>>>>> speed test should have a specified set of positions which
>>represents
>>>>> a typical match.
>>>>> >
>>>>> > -Øystein
>>>>> >
>>>>> Yes, we ought to have an analysed match, a few money games, and a
>>>>> few rolled out positions for people to test speed and
>>correctness on.
>>>>
>>>> Having something close to a non-regression test would be just great.
>>>> First idea would be to have it embedded into gnubg's code,
>>maybe as a
>>>> python script.
>>>>
>>>> The current speed test (withot the bug) is fine enough to
>>measure the
>>>> eval speed (which is not the overall speed you can expect
>>in a rollout).
>>>> For example, it gives you quickly an idea of how much a -O0
>>build is
>>>> slower than a -03 one. I'm more for fixing it and work on a
>>separate
>>>> non-regression part.
>>>>
>>>> MaX.
>>>>
>>>> _______________________________________________
>>>> Bug-gnubg mailing list
>>>> address@hidden
>>>> http://lists.gnu.org/mailman/listinfo/bug-gnubg
>>>>
>>>>
>>>
>>>
>>> _______________________________________________
>>> Bug-gnubg mailing list
>>> address@hidden
>>> http://lists.gnu.org/mailman/listinfo/bug-gnubg
>>>
>>
>>
>>_______________________________________________
>>Bug-gnubg mailing list
>>address@hidden
>>http://lists.gnu.org/mailman/listinfo/bug-gnubg
>>
>
>
> -------------------------------------------------------------------
> The information contained in this message may be CONFIDENTIAL and is
> intended for the addressee only. Any unauthorised use, dissemination of the
> information or copying of this message is prohibited. If you are not the
> addressee, please notify the sender immediately by return e-mail and delete
> this message.
> Thank you.
>

python-scripting-sysarg.patch
Description: Binary data