[PATCH] {master} automake: new option 'serial-tests'

[Stefano Lattarini]

Currently, automake offers a 'parallel-tests' option to enable the
use of the parallel testsuite harness in the generated Makefiles,
but no option to explicitly state the intention of using the older
serial testsuite driver (which is currently the default).

This makes the parallel test harness seems like a second-class
citizen (while actually it should be the other way around); more
importantly, it will prevent us from making the parallel harness
be the default one in future automake versions, since the users
of the serial harness would then have no way to instruct automake
to continue using it.

We solve all of this by introducing a new 'serial-tests' option.

* lib/Automake/Options.pm (_process_option_list): Recognize the
new option, and related minor refactoring.
* tests/serial-tests.test: New test.
* tests/list-of-tests.mk: Add it.
* doc/automake.texi: Update.
* NEWS: Likewise.
---

I will push in a couple of days if there is no objection.

 NEWS                    |    5 +++
 doc/automake.texi       |   24 +++++++++----
 lib/Automake/Options.pm |   11 +++++-
 tests/list-of-tests.mk  |    1 +
 tests/serial-tests.test |   86 +++++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 119 insertions(+), 8 deletions(-)
 create mode 100755 tests/serial-tests.test

diff --git a/NEWS b/NEWS
index 4c7a54a..aef4626 100644
--- a/NEWS
+++ b/NEWS
@@ -30,6 +30,11 @@ New in 1.11a:
 
 * Changes to Automake-generated testsuite harnesses:
 
+  - The new automake option 'serial-tests' has been introduced.  It can
+    be used to explicitly instruct automake to use the older serial
+    testsuite harness.  This is still the default at the moment, but it
+    might change in future versions.
+
   - Test scripts that exit with status 99 to signal an "hard error" (e.g.,
     and unexpected or internal error, or a failure to set up the test case
     scenario) have their outcome reported as an 'ERROR' now.  Previous
diff --git a/doc/automake.texi b/doc/automake.texi
index f342686..3e6a2bf 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -8927,15 +8927,18 @@ by the tests, not the tests themselves.  Of course you 
can set
 
 @node Serial Test Harness
 @subsection Serial Test Harness
address@hidden @option{serial-tests}, Using
 
 @emph{NOTE:} This harness, while still being the default one, is
-obsolescent, and kept mostly for backward-compatibility reasons.
-The user is advised to use the parallel test harness instead
-(@pxref{Parallel Test Harness}).
-
-The serial harness operates by simply running the tests serially, one at
-the time, without any I/O redirection.  It's up to the user to implement
-logging of tests' output, if that's requited or desired.
+obsolescent, and kept mostly for backward-compatibility reasons.  The user
+is advised to use the parallel test harness instead (@pxref{Parallel Test
+Harness}).  Be warned that future Automake versions might switch to use
+that more modern and feature-rich harness by default.
+
+The serial test harness is enabled by the Automake option
address@hidden It operates by simply running the tests serially,
+one at the time, without any I/O redirection.  It's up to the user to
+implement logging of tests' output, if that's requited or desired.
 @c TODO: give an example of how this can be done.
 
 For historical and implementation reasons, the @code{AM_TESTS_ENVIRONMENT}
@@ -10139,6 +10142,13 @@ this directory.
 Enable test suite harness for @code{TESTS} that can run tests in parallel
 (@pxref{Parallel Test Harness}, for more information).
 
address@hidden @option{serial-tests}
address@hidden Option, @option{serial-tests}
address@hidden serial-tests
+Enable the older serial test suite harness for @code{TESTS} (@pxref{Serial
+Test Harness}, for more information).  This is still the default for the
+moment.
+
 @item @option{readme-alpha}
 @cindex Option, @option{readme-alpha}
 @opindex readme-alpha
diff --git a/lib/Automake/Options.pm b/lib/Automake/Options.pm
index 642f337..eee76db 100644
--- a/lib/Automake/Options.pm
+++ b/lib/Automake/Options.pm
@@ -276,6 +276,15 @@ sub _process_option_list (\%@)
           error ($where, "support for lzma-compressed distribution " .
                          "archives has been removed");
         }
+      elsif ($_ eq 'parallel-tests')
+        {
+          # Just recognize it explicitly.
+        }
+      elsif ($_ eq 'serial-tests')
+        {
+          # This is a little of an hack, but good enough for the moment.
+         delete $options->{'parallel-tests'};
+        }
       elsif ($_ eq 'no-installman' || $_ eq 'no-installinfo'
             || $_ eq 'dist-shar' || $_ eq 'dist-zip'
             || $_ eq 'dist-tarZ' || $_ eq 'dist-bzip2'
@@ -286,7 +295,7 @@ sub _process_option_list (\%@)
             || $_ eq 'subdir-objects' || $_ eq 'nostdinc'
             || $_ eq 'no-exeext' || $_ eq 'no-define'
             || $_ eq 'std-options'
-            || $_ eq 'color-tests' || $_ eq 'parallel-tests'
+            || $_ eq 'color-tests' 
             || $_ eq 'cygnus' || $_ eq 'no-dependencies')
        {
          # Explicitly recognize these.
diff --git a/tests/list-of-tests.mk b/tests/list-of-tests.mk
index d10f585..f19c930 100644
--- a/tests/list-of-tests.mk
+++ b/tests/list-of-tests.mk
@@ -688,6 +688,7 @@ override-suggest-local.test \
 parallel-am.test \
 parallel-am2.test \
 parallel-am3.test \
+serial-tests.test \
 parallel-tests.test \
 parallel-tests2.test \
 parallel-tests3.test \
diff --git a/tests/serial-tests.test b/tests/serial-tests.test
new file mode 100755
index 0000000..eacdd61
--- /dev/null
+++ b/tests/serial-tests.test
@@ -0,0 +1,86 @@
+#! /bin/sh
+# Copyright (C) 2009-2012 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Option 'serial-tests'.
+
+# To avoid useless generation of a sibling test.
+am_parallel_tests=yes
+am_create_testdir=empty
+. ./defs || Exit 1
+
+hasnt_parallel_tests ()
+{
+  $EGREP 'TEST_SUITE_LOG|TEST_LOGS|\.log.*:' $1 && Exit 1
+  grep 'recheck.*:' $1 && Exit 1
+  grep '^check-TESTS: \$(TESTS)$' $1
+}
+
+has_parallel_tests ()
+{
+  $EGREP '(^| )check-TESTS.*:' $1
+  $EGREP '(^| )recheck.*:' $1
+  grep '^\$(TEST_SUITE_LOG): \$(TEST_LOGS)$' $1
+  grep '^\.test\.log:$' $1
+}
+
+mkdir one two
+
+cat >> one/configure.ac <<END
+AC_INIT([$me], [1.0])
+AM_INIT_AUTOMAKE([serial-tests])
+AC_CONFIG_FILES([Makefile])
+END
+
+echo 'TESTS = foo.test bar.test' > one/Makefile.am
+
+cat >> two/configure.ac <<END
+AC_INIT([$me], [2.0])
+AC_CONFIG_AUX_DIR([config])
+AM_INIT_AUTOMAKE([parallel-tests])
+AC_CONFIG_FILES([aMakefile bMakefile])
+END
+
+cp one/Makefile.am two/aMakefile.am
+cat - one/Makefile.am > two/bMakefile.am <<END
+AUTOMAKE_OPTIONS = serial-tests
+END
+
+cd one
+touch missing install-sh
+$ACLOCAL
+$AUTOMAKE
+grep TEST Makefile.in # For debugging.
+hasnt_parallel_tests Makefile.in
+test ! -r test-driver
+cd ..
+
+cd two
+mkdir config
+$ACLOCAL
+$AUTOMAKE --add-missing
+grep TEST [ab]Makefile.in # For debugging.
+has_parallel_tests aMakefile.in
+hasnt_parallel_tests bMakefile.in
+mv aMakefile.in aMakefile.sav
+mv bMakefile.in bMakefile.sav
+test ! -r test-driver
+test -f config/test-driver
+$AUTOMAKE
+diff aMakefile.sav aMakefile.in
+diff bMakefile.sav bMakefile.in
+cd ..
+
+:
-- 
1.7.9