Automatic tests

[Andrei Kholodnyi]

>> > A good example is the Python API tests in
>> > ?src/api/python/speechd/_tests.py
>> does it make sense to move it under src/tests/api/python?
>
> I don't see a reason for that. The script that runs tests
> should run the appropriate tests from all submodules,
> including the Python API.

this is a problem of structure.
do we want to have e.g. src/api/python and tests/api/python
or src/api/python and src/api/python/tests

I'm fine with both, but it would be nice if we agree on one or another.
IMO tests should be grouped similar.

>> This I didn't get, why do you think that most of the tests are
>> installation dependent?
>
> Most anything that has to do with speech has to do with
> TTS systems and audio backends, both of them are external
> tools.
>
> So what I would like to be able to do is to
> ? ?1) Generate tarball/package
> ? ?2) Install on several virtual machines with different configurations
> ? ?3) Run autotests script to evaluate

agree.

> Thus I think a separate script is most versatile, while
> make target or even make distcheck is too specific.

with autotest you have this script called testsuite
it allows you to run individual tests or any combination, manually or automated.
make rules just starts this script, nothing more than that:
$(SHELL) $(TESTSUITE) AUTOTEST_PATH="$(bindir)" $(TESTSUITEFLAGS)