test infrastructure [was: suggested workflow for macro testing?]

[Francesco Salvestrini]

Hi all,

I just grabbed the same test-environment I use in my projects (that layout is 
in use both in savannah and in github ones) and slashed it until it worked 
into the archive.

Don't blame me for the final result, some work is still needed and I have not 
enough time today to clean up the work as I wish.

We will need to clean (and fix as well) it to suite our needs but It should be 
a good starting point anyway.

Hack it the way you like, ask for details if needed.

IMHO:

1) Autotest would be better for us (in order to handle inter-test dependencies 
as well) but we could start from this approach, changing it later to 
autotest.

2) I think that we should point to have a good set of regression tests as a 
first step then change the approach later. The change related efforts should 
be little.

Side notes:

* I used the license form used in other files in the maint branch even if it 
seems to me that we should use the longer form. Dunno.

* Use 'make check -C tests' for plain tests. 'VERBOSE=x make check -C tests' 
to enable verboseness. 'DEBUG=x make check -C tests' should be used (not yet 
supported) to enable debugging into autoconf and aclocal.

* test files should start with test-xxx.in in order to not pollute the tests 
directory

* If more test per macro-x then test-macro-x-1.in, test-macro-x-2.in, 
test-macro-x-3.in names should be used

* tests/Makefile.am should be rearranged to avoid targets repetition. I 
sticked to the portable way (not all our users use gmake).

* The test-x.in gets executed into tests/testSubDir-x. It should be 
straightforward to check output files there than polluting the tests 
directory. E.g: This way test-x-1, producing file AAA.txt, cannot overwrite 
AAA.txt coming from test-y-12.

* The 'required' variable should be filled with the requirements the test 
needs. In a requirement miss, the test should be automatically skipped by 
automake.

Have fun,
Francesco

-- 
Do not underestimate the value of print statements for debugging.

0001-Introduced-autoconf-like-tests-regression-environmen.patch
Description: Text Data