Automatic tests

[Andrei . Kholodnyi]

> They are however, not really any good for automated
> testing. The kind of tests that belong to make distcheck are
> 1) automatic non-interactive tests
> 2) interactive tests with clear yes/no resolution

I think we need to analyze them and convert to automated when applicable.

> I think the issue of comparing the sound output
> with expected is much too difficult in all its relations
> and we do not really need that.

I'd put is as a long term goal.

> We would profit
> highly if eg we had simple tests that run through
> all the available SSIP commands (as described in
> SSIP documentation) and check their replies

agree, however there is c api which need to be tested as well.

> A good example is the Python API tests in
> src/api/python/speechd/_tests.py

does it make sense to move it under src/tests/api/python?

> A set of interactive tests (eg priority resolution)
> with a clear pass/fail result that needs to be
> confirmed by the user are also possible. I would
> appreciate having them aligned in a script one
> after another.

I think they could be automated as well,
eg you can wait for particular notifications /cancel, begin etc/ and  
analyze them.

> As for placement: I think only few tests are installation
> independent and make sense as part of make distcheck.
> I think most useful would be a simple script or some special
> make target, that one could run manually *after* installing
> the .tar.gz or even distribution package, as on various hosts
> in virtual machines, in various setups (user vs. system wide) etc.

This I didn't get, why do you think that most of the tests are installation  
dependent?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: 
<http://lists.freebsoft.org/pipermail/speechd/attachments/20100930/cc73c4e8/attachment.htm>