[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: serial-tests option and backwards compatibility
From: |
Stefano Lattarini |
Subject: |
Re: serial-tests option and backwards compatibility |
Date: |
Mon, 04 Feb 2013 14:31:43 +0100 |
On 02/04/2013 01:16 AM, Luke Mewburn wrote:
> On Wed, Jan 16, 2013 at 12:12:13PM +0100, Stefano Lattarini wrote:
> | On 01/15/2013 07:32 PM, Brandon Black wrote:
> | > So my conundrum is this: in automake-1.13, the default switched from
> serial
> | > testing to parallel testing. The only way I can see to disable this is
> to
> | > add "serial-tests" to my AM_INIT_AUTOMAKE. However, my AM_INIT_AUTOMAKE
> | > also only requires version 1.11.6, which I'd like to continue
> supporting,
> | > and these older versions of automake will barf on the invalid option
> | > "serial-tests" (I also tried "no-parallel-tests", but that's also
> invalid).
> | >
> | Yes, the "serial-tests" option was unfortunately only introduced in
> | Automake 1.12.
>
>
> We use autotest extensively, and our test scenario is rather complex
> and relies upon the existing serial behaviour;
>
I don't follow; if you use autotest, why you also need to rely on the
AUtomake-provided test harnesses? None of the GNU projects using
autotest and that I know of (tar, bison, autoconf, libtool) does so.
> we setup file systems
> and databases, we start daemons, etc. We can't use parallel tests
> at this time.
>
> If parallel tests supported a dependency graph a la make
>
But it does; in fact, it relies on make to honour such a dependency graph;
quoting from section 15.2.3 "Parallel Test Harness" of the Automake manual
<http://www.gnu.org/software/automake/manual/automake.html#Parallel-Test-Harness>:
In order to guarantee an ordering between tests even with make -jN,
dependencies between the corresponding .log files may be specified
through usual make dependencies. For example, the following snippet
lets the test named foo-execute.test depend upon completion of the
test foo-compile.test:
TESTS = foo-compile.test foo-execute.test
foo-execute.log: foo-compile.log
Please note that this ordering ignores the results of required tests,
thus the test foo-execute.test is run even if the test foo-compile.test
failed or was skipped beforehand. Further, please note that specifying
such dependencies currently works only for tests that end in one of
the suffixes listed in TEST_EXTENSIONS.
Or am I missing something else?
> or tsort then we could possibly take advantage of that.
>
>
> In my humble opinion, providing an option to control a feature in
> a major release (1.12.x) then change the default for that feature
> in the next major release (1.13.x) 7 months later is too short a period.
>
With this I agree, in retrospect. But it's too late to change that
(reverting the default-switching only to re-introduce it in a later
release would just exacerbate confusion at this point, IMHO).
> Especially when the time between previous major releases was 2.5 years.
>
> Examining the Changelog and release dates:
>
> 2009-12-08 Release automake 1.11.1
>
> 2012-02-21 Add "serial-tests" support (in "HEAD")
>
> 2012-04-13 Release automake 1.11.5 (without "serial-tests")
>
> 2012-05-18 Parallel tests now the default (in "HEAD", not 1.11.x)
>
> 2012-06-01 Release automake 1.12.1 (with "serial-tests")
>
When I did this, I should really have published a 1.11.x release offering
this same option as well; that would have removed all confusion. Sigh,
such a low-hanging fruit not picked :-(
> 2013-01-01 Release automake 1.13.1 (parallel tests now default)
>
>
> This isn't the only backwards incompatible change made recently,
> and in my humble opinion I think the timeframes introducing
> backwards incompatibility are too aggressive.
>
You are not the only one to think so, and I've come to agree (at least
partially); for some more discussions and background, see:
<http://lists.gnu.org/archive/html/automake/2013-02/msg00001.html>
<http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13578>
So things should proceed more carefully in the future (I hope).
> From a backwards-compatibility point of view, I think the default
> should be reverted to serial tests, and make it clearer that
> parallel tests are available as an option.
>
With this I must disagree, sorry.
Thanks,
Stefano