[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: parallel-tests: redo lazy checking: recheck and RECHECK_LOGS.

From: Akim Demaille
Subject: Re: parallel-tests: redo lazy checking: recheck and RECHECK_LOGS.
Date: Wed, 1 Apr 2009 16:53:28 +0200

Le 31 mars 09 à 01:01, Ralf Wildenhues a écrit :

Salut Ralf !

* Akim Demaille wrote on Mon, Mar 30, 2009 at 09:50:59PM CEST:
Le 29 mars 09 à 00:04, Ralf Wildenhues a écrit :

I often use


which makes all failing test strict.  In other words, successful
tests are not rerun by "make check", but failing tests are. This is
because our test suite sometimes hits the limit of the machine, and
tests timeout for no sound reason.  So there is no reason to rerun
successful tests, but failing test might pass if the machine has a
lesser load.

This is what the 'recheck' target does, right?

Not exactly. make recheck runs only the tests that failed, while here, in addition, the tests whose dependencies have changed will be run again.
Not to mention test that where not run at all.

I don't think your interpretation is right.  Those tests whose
dependencies have changed will be rerun in any case, with your
recheck, with my recheck, with above.  You cannot prevent it unless
you remove those dependency statements from the makefile.

Err, really?  My code reads:

# Get the list of tests that failed.
LIST_FAILED_TEST_LOGS =                                                 \
  perl -e 'print join " ",                                            \
                 grep {open(_) && <_> =~ /^(FAIL|XPASS): /} (@ARGV);'     \
       -- $(TEST_LOGS)
.PHONY: recheck recheck-html
recheck recheck-html:
        @target=$$(echo $@ | sed -e 's/^re//');                 \
        if test -f $(TEST_SUITE_LOG); then                      \
          TESTS=$$($(LIST_FAILED_TEST_LOGS));                   \
          $(MAKE) $(AM_MAKEFLAGS) $$target TESTS="$$TESTS";   \
        else                                                    \
          $(MAKE) $(AM_MAKEFLAGS) $$target;                     \

So I really get the list of logs that were FAIL'd, remove them, and run them as TESTS. Here I'm using the fact that TESTS accepts *.log in my implementation. I don't see why logs that passed but are obsoleted would run again.

I have not understood exactly what you are referring too, but why don't
we just pass TESTS=$(TESTS) explicitly?

Because it can cause command line length issues.  Because variable
overriding is inherently fragile: it can subtly change the meaning
of the underlying makefile.

When another indirection for globbing were added, we'd have to pass
$(TESTS) through more than one $(MAKE).  And we'd have to pass
$(TEST_LOGS) as well, if we were to allow globbing on that.  And
I would like to avoid passing both, that halves our available length

Wow. What's the worse case for this length? How about having another mode to run the test suite, based on a temporary MANITEST file with the list of files to run? Maybe a file and a 'make -f check'? It would have list the dependencies from test- suite.log to the individual logs, and include the main Makefile. I dunno, maybe I don't make any sense here, or maybe there are complications I have no foreseen.

Are there non-GNU Makes that support -j?

Or ask the users to add his "precious macros" to AM_MAKEFLAGS?

We could do that. In fact, I intend to add a note to this extent to the
Autoconf manual section about `make' portability, but AM_MAKEFLAGS
should be mentioned in automake.texi too I guess. What I don't know is whether it was intended as user variable or as author realm.

Given it's name, I think it should be userland. __AM_ whatever is a different beast, but AM_CFLAGS and so forth are definitely userland.

Can't we have it when using GNU Make? Can't we have some sort of #ifdef
HAVE_GNU_MAKE sections in the output Makefile?  I'm not referring to
AM_CONDITIONAL, but to some nice run-time trick.

Produce a GNUmakefile in the build directory.  Let this GNUmakefile
include the Makefile.  Put all your nice goodies in GNUmakefile.

An example of this is what the gnulib module gnumakefile does.
For example, it allows you to
 make -j3 clean all check distcheck

nice :)

and it will work!  In a git repo, and with the git .version strategy
used by several gnulib-using packages, it will autoreconf so the tarball
will have an up to date version string.

Yes, I also have some GNUmakefiles where I want to play that kind of trick. But this kind of clashes with users that already do. I was looking for something for Automake, not for the Automake user.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]