Re: Parallel tests execution [0/4]

[Akim Demaille]

Le 16 oct. 08 à 22:02, Ralf Wildenhues a écrit :

> Thanks.  FWIW, it still has some of the portability issues that I
> mentioned.

Yes, I know, but we don't aim maximum portability.  For a start, we  
expect GNU Make.  But I'll be happy to use Automake's version to gain  
portability :)


> > > - (5), (6), (8) are provided already by the check.mk code, except  
> > > that
> > >  (8) didn't work.
> >
> > Actually I never meant to have hard error stop the whole test suite.
> > The point of hard-errors as they were defined in check.mk was to make
> > them *non* ignorable.  For instance our test suite raises a hard- 
> > error
> > if the program make a segmentation fault, which we never want to
> > tolerate.
>
> I don't understand.  What is the difference to a normail FAIL then,
> i.e., to the process exiting with 1?

By non-ignorable, I meant cannot be XFAILed.  It was meant for error  
that should never be accepted.

At some point we were developping more-or-less test driven: many tests  
were written way before it was possible to pass them, so we knew that  
the tests would fail and they were XFAIL'ed.  Yet, it was not  
acceptable for the executable being tested to die with a SEGV.  So  
simple failures are just exit 1, and hard-errors (or 'unacceptable  
errors') had an exit status that could not be saved by XFAIL.


> > > - output `PASS: foo.test' not `PASS: foo.log'
> >
> > This was actually a feature :) We use an Emacs mode that opens the
> > (log) file when we click on it.
>
> But it's not the log file that fails.  I found this very non- 
> intuitive.

Ok.

> I might be talked into a compromise, though; for example like this:
>  FAIL: sub/foo.test (see sub/foo.log)

Good with me too.


> > > - how should I best acknowledge The Vaucanson Group?  Something like
> > > this in lib/am/check.am ok?
> >
> > > ## This code is adapted from check.mk which came from:
> > > ##
> > > ## Vaucanson, a generic library for finite state machines.
> > > ## Copyright (C) 2006, 2007 The Vaucanson Group.
> >
> > Actually it would be more fair to thanks EPITA and Gostai, both  
> > worked
> > on it, and Vaucanson was just the initial impetus to develop this.
>
> OK, I will use this:
>
>  ## This code is adapted from check.mk which was originally
>  ## written by The Vaucanson Group, further developer at
>  ## EPITA and Gostai, then made its way from GNU coreutils
>  ## to end up, largely rewritten, in Automake.

Well, it was developped by me for Vaucanson, and we usually hide our  
names behind "The Vaucanson Group".  That was part of my work at  
EPITA.  So I would completely drop the Vaucanson group from there.   
How about this?

>  ## This code is adapted from check.mk which was originally
>  ## written at EPITA/LRDE, further developer at Gostai, then made
>  ## its way from GNU coreutils to end up, largely rewritten,
>  ## in Automake.