Re: Unittests

[BVK Chaitanya]

Hi,

On Thu, Dec 17, 2009 at 5:15 AM, Carles Pina i Estany <address@hidden> wrote:
>
> when I said "it's very easy" I was thinking more like "you explain to me
> how the unittest infrastructure works and I write the test" more than
> the other way. But no problem at all if you want to write it!
>

For scripted tests, unit test framework simply executes the script and
reports PASS/FAIL based on its exit status.  Attached patch has an
example_scripted_test.in

Basically, add a testcase.in (.in would make it run through autoconf
stuff, so you can use @builddir@, @srcdir@, etc. stuff in your
testcase) file into tests/ subdir and modify conf/tests.rmk file to
include your testcase, as

check_SCRIPTS += testcase
testcase_SOURCES = tests/testcase.in

and run configure and "make check".  Let me know if you face any problems.

> This is not what I wanted to test using the unittest, but I can show a
> shell script that I did last june testing for these things.
>
> address@hidden:~/grub2/oldgrub/grubprova$ cat test.sh
> #!/bin/bash
>
> export TEXTDOMAIN="testscript"
> export TEXTDOMAINDIR="./locale"
> echo $(gettext "hello world")
>
> And then:
> address@hidden:~/grub2/oldgrub/grubprova$ find
> .
> ./locale
> ./locale/ca
> ./locale/ca/LC_MESSAGES
> ./locale/ca/LC_MESSAGES/testscript.mo
> ./locale/address@hidden
> ./locale/test.pot
> ./test.sh
> address@hidden:~/grub2/oldgrub/grubprova$
>


Currently, GRUB-2 build system doesn't provide this directory
structure, we need to fix it :-(


> But what I was thinking to test with the unittest is the gettext.mod,
> not outside. I could do semi-manually doing:
> a) Different .mo files with some strings (inside /boot/grub/locale/[.mo
> files])
> b) A Grub2 disk image with the script to check the gettext module.
> Mainly would do (I'll write in pseudo-Bash, would be in Grub script):
> -------
> set lang=ca
> TESTFAILED=false
> if [ $(gettext "hello world") -ne "hola món" ]
> then
>        echo "Test failed 01!"
>        TESTFAILED=true
> fi
>
> if [ ... ]
> then
>        echo "Test failed 02!"
>        TESTFAILED=true
> fi
>
> set lang=es
>
> if [ ... ]
> then
>
> fi
>
> if [ $TESTFAILED -eq "true" ]
> then
>        echo "TESTS FAILEDS"
> fi
> -------
>
> So when I do changes in gettext module (like optimizse something, add
> support for plurals, etc. etc.) I would be sure that everything is all
> right.
>
> Was you thinking something like this?
>

Well, not exactly!  Since $(gettext "hello world") output must be same
in grub-shell environment and regular bash-shell, we can automate most
of the test case failures.  For example, for below script, we expect
the output to be same both on grub-shell and bash-shell

#! grub-tester
gettext "hello world"
LANG="zh_CN" gettext "helloworld"

so *if* we can write an interpreter (like, grub-tester above) that
executes given script in both shells and compares the output
automatically, our unit tests would be a lot easier.  A testcase
failure should be raised by grub-tester if it finds that grub-shell
output is different than bash-shell output for that script :-)  Of
course, testcase should use only those commands that are valid in both
grub and bash.

For this to work we need a script to start grub image inside qemu with
user specified script and capture its output.

   Earlier I thought we could use grub-qemu port, but i am now
convinced that grub-mkrescue tool is sufficient.  I am half way in
doing this already.  I hit these below problems, but i have
workarounds in mind:

  a.  Option for not to display grub prompt (add PS1 environment variable ???)
  b.  Option to set noecho flag to tty.
  c.  Option to disable NCURSES escape sequences for serial terminal
(add TERM=dumb support ???)

Currently I am trying to cleanup escape sequences, grub> prompt and
echoed characters from serial port output captured from grub;
hopefully it should do for some time.



regards,
-- 
bvk.chaitanya

unit-testing-framework-on-exp-1432.patch.txt
Description: Text document