automake
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: serial-tests option and backwards compatibility


From: Stefano Lattarini
Subject: Re: serial-tests option and backwards compatibility
Date: Thu, 17 Jan 2013 14:39:07 +0100

On 01/16/2013 02:39 PM, Milan Broz wrote:
> On 01/16/2013 12:12 PM, Stefano Lattarini wrote:
>> On 01/15/2013 07:32 PM, Brandon Black wrote:
>>> So my conundrum is this: in automake-1.13, the default switched from serial
>>> testing to parallel testing.  The only way I can see to disable this is to
>>> add "serial-tests" to my AM_INIT_AUTOMAKE.  However, my AM_INIT_AUTOMAKE
>>> also only requires version 1.11.6, which I'd like to continue supporting,
>>> and these older versions of automake will barf on the invalid option
>>> "serial-tests" (I also tried "no-parallel-tests", but that's also invalid).
>>>
>> Yes, the "serial-tests" option was unfortunately only introduced in
>> Automake 1.12.
>>
>>> There doesn't seem to be a way to set up my configure.ac such that my tests
>>> will run in serial mode on both older and newer versions of automake
>>>
>> Since the parallel test harness is already present in Automake 1.11, and
>> it is in many many ways superior to the old serial driver, my suggestion
>> would be switch to in your package.  But assuming you have good reasons
>> not to do so ...
>>
>>> (1.11.6 -> 1.13) without causing an automake error (invalid option) in
>>> 1.11.6.  Am I missing some tricky method of working around this, or is it
>>> just not possible?
>>>
>> ... you might try a trick like this in your configure.ac:
>>
>>   AM_INIT_AUTOMAKE(
>>     1.11.1
>>     dnl Automake version before 1.13 (when the serial-tests option was
>>     dnl still the default) still defined the badly obsolete macro
>>     dnl 'AM_PROG_INSTALL'.
>>     m4_ifndef([AM_PROG_INSTALL], [serial-tests])
>>   )
> 
> Hm. This doesn't seem to work for me (at least not on Fedora, seems like
> it sees the macro still defined).
>
Anther simpler (albeit less "automagical" solution) would be to have a
bootstrap script that deals with this minor incompatibilities; something
like (untested):

  $ cat bootstrap
  am_ver=`$AUTOMAKE --version | sed -1p`
  # Autmake 1.11.x doesn't grasp the 'parallel-tests' option.
  case $am_ver in
    *\ 1.11*|*\ 1.12*) echo 'm4_define([PARALLEL_TESTS_OPTION], [])';;
    *) echo 'm4_define([PARALLEL_TESTS_OPTION], [parallel-tests])';;
  esac > pt-opt.m4
  ${AUTORECONF-autoreconf} -fvi

  $ cat configure.ac
  AC_INIT([me], [1.0])
  m4_include([pt-opt.m4])
  AM_INIT_AUTOMAKE(1.11.1 foreign PARALLEL_TESTS_OPTION)
  ...


> What's worse, as a side effect of the parallel tests harness default,
> all test log output is now sent to log file instead to standard output.
> 
That is by design.  If you want to send output to the terminal, you
can do so though a proper use of the AM_TESTS_FD_REDIRECT variable
(documented in the manual).  Well, you can do that since Automake 1.12
only, actually.  If you want to support a similar feature in 1.11 too,
you'll need the same other hack with TESTS_ENVIRONMENT (refer to the
GNU Coreutils testsuite for an example).

> I have computational intensive tests  which print some status to
> show user progress - it is now gone with 1.13.x.
> It prints only final status, keeping screen blank for minutes.
> So I definitely need some working hack as above to support
> old and new automake versions...
> 
> (I know future is to use TAP log protocol, but that's not an excuse
> to break working package tests.)
>
Actually, TAP makes sense in a lot of situations, but not always,
an maybe not even most of the time.  First, several TAP tests in
a single script are usually run serially (most TAP-producing libraries
out there are not multi-threaded), and on fast and many-cores machines
this can be a real performance killer if you implement your testsuite
in just a handful of TAP scripts.  In addition, TAP is overkill for
many testing scenarios, not worth the trouble.  The good thing of the
parallel Automake harness is that it allows you to freely mix "classic"
test scripts and TAP based test script, allowing you to use the best
of both worlds (Automake' own test suite makes an heavy use of this
capability).

I'd like to stress that the usefulness of the parallel harness transcends
(and in fact, pre-dates) the introduction of TAP support in Automake.
Its killer features where the automatic logging of test output, easy
parallelization of test runs, and the "recheck" target.

Regards,
  Stefano



reply via email to

[Prev in Thread] Current Thread [Next in Thread]