Re: [pdf-devel] Test Specification Document

[Brad Hards]

On Monday 25 February 2008 07:08:19 am address@hidden wrote:
> Hi.
>
> I introduced a new texinfo document in `doc/' named `gnupdf-tsd.texi'
> (for GNU PDF Library Test Specification Document). It is the place
> where we will document all the unit tests, subsystem tests and system
> tests of the library.
Is this really worth while? It seems quite imprecise - example:

pdf_realloc 
Test: pdf_realloc_001  
Reallocate some bytes. 
Success condition  
The call should not produce an error.

It would be better to express some initial conditions, a set of interesting 
variations (realloc smaller, realloc same size, realloc larger), and then 
check it actually worked (not just that the error handling is turned off :-))

At this level, expressing the test in C would seem better.

I am not saying that there shouldn't be a test plan / spec, just that it 
should be at a higher level - expressing goals rather than detail:

* Goal: all functions should have unit tests that verify correct operation.
* Goal: all functions should have unit tests that check bad parameter 
combinations result in meaningful errors.
* Goal: subsystems shall...
* Goal: systems shall...

Then write the unit tests in well commented, well structured code. Ideally use 
a test harness that provides a bit of expressive power - I've used "check" 
and it seems OK. CppUnit is better, as long as you are willing to tolerate 
C++.

The test plan / spec could describe the test harness, and related tools (e.g. 
unit tests will be run on check-in using some kind of continuous integration 
tool; tests will be run nightly in valgrind; etc).

I have some ideas for writing interactive tests, but it seems this project has 
a way to go before that will be important, and I'm still developing my ideas, 
so enough for now :-)

Brad