Re: Autotest: loops of tests

[Noah Misch]

On Tue, Mar 22, 2005 at 10:05:04AM +0100, Ralf Wildenhues wrote:
> * Noah Misch wrote on Sat, Mar 19, 2005 at 10:29:32PM CET:
> > I suggest using an m4sugar looping construct or writing a macro to abstract 
> > the
> > common bits.  AT_CHECK_AT_TITLE_CHAR and its underlying stack in autotest.at
> > demonstrate the latter approach.  Does one of those techniques meet your 
> > needs?
> 
> I think they would work, and for small data sets, they are fine.
> I'm not too happy with the replication they cause in `testsuite' -- one
> of my ideas was to test on the order of 2^8 link flag combinations
> eventually, and I would like to keep the blow-up as small as possible.

I understand.  Current Autotest does not provide a clean way to avoid that
bloat.  That is a problem worth addressing; the damage to non-Autoconf users of
Autotest is particularly bad since you distribute `testsuite' itself.

I have contemplated adding a macro that runs shell commands in the top-level
`testsuite' shell, allowing one to do global variable initializations.  You
could then emit the text of the test case once, for a variable assignment, and
reference the variable in the replicated bits of your test.  That would reduce
the bloat somewhat, and it is easy to implement.

We could even optimize your example transparently by noticing duplicate AT_DATA
calls and storing the text only once in `testsuite'.  That would also help parts
of the Autoconf test suite.

> I understand that having both m4 constructs and shell loops work
> together is quite difficult.

Mixing _diversions_ with shell control structures is difficult.  M4 and the
shell generally interact well since they both process input in a linear fashion.
When you change that by using a branching construct like an M4 diversion or a
shell function, that synergy breaks down.

Autotest uses diversions more than does Autoconf.  In particular, it diverts
every AT_SETUP ... AT_CLEANUP block into one branch of a massive case statement.
All text outside of any AT_SETUP ... AT_CLEANUP is divert(KILL).

The matters you raise are important.  Thank you.