Re: more Autotest thoughts

[Noah Misch]

Hi Ralf,

On Fri, Apr 22, 2005 at 09:16:48AM +0200, Ralf Wildenhues wrote:
> It would be nice if autotest put each test (or test group) into a
> separate file, and then source or exec it in a subshell.  This is

I once considered doing this as part of a scheme to run several tests in
parallel.

> faster when all you do is `testsuite -v 123'.  Libtool's small test
> suite takes noticeable time to skip over 10 tests.  Old shells with
> link-farming for here documents will take a very long time because of
> the huge case statement.  Is there an m4 limitation that prevents a
> fix for this?

M4 has no support for creating multiple output files, but it would be easy to
mark file boundaries in output via magic separators and do the splitting in
autom4te or another, dedicated script.

> Furthermore, the AT_CHECK docs are not fully clear to me: If COMMANDS
> fail, but RUN_IF_FAIL is given, then AT_CHECK still only acts upon the
> COMMANDS exit status.  In other words: it is not possible to change
> the exit status of the test group from within RUN_IF_FAIL (thinking
> this may possible may be a flaw of mine -- or suboptimal docs).  Also,

The existing behavior makes sense to me.  How could we improve the semantics or
documentation?

> the interaction of COMMANDS exit status and STDOUT and STDERR
> expectations are IMHO not described accurately.  Namely, if COMMANDS
> exit with 77, but also produce unwanted output, the test fails.  This
> is a valid, but unfortunate combination for me, as it means I might
> have to discard or ignore output.

Can you post a sample test group that this upsets?

> AT_DATA doc should include the fact that no shell expansion will take
> place on the second argument, but on the first.  I would like a macro

Yes, all of these facts about AT_DATA should be documented.

> similar to AT_DATA which shell-expands its second argument.  But
> frankly, I really don't see the advantage of either command over just
> using
> 
> | for i in 1 2; do
> |   cat >expanded_file$i <<EOF

Use `cat' and friends.  I see no need for a special macro that populates a file.

> The apparent beauty of m4 macros seems IMVHO to be traded for increased
> output size more often than is technically necessary.

Which macros?