Re: [PATCH] Testsuite: use $SHELL to run tests which are shell scripts.

[Stefano Lattarini]

On Thursday 02 September 2010, Stefano Lattarini wrote:
> > The patch has obvious advantages though.  How to best ameliorate
> > the disadvantages?
> 
> Hmm... maybe by making `tests/defs' re-execute the current test
> with $CONFIG_SHELL unless a proper environment variable (let's say
> `AM_TESTSUITE_NO_REEXEC') is set?
> Then we can set AM_TESTSUITE_NO_REEXEC in TEST_ENVIRONMENT (or
> its equivalent under `parallel-tests' option), to avoid useless
> re-execs.
Done something on these lines in the attache patch.

OK for the `tests-defs' branch?
 
> I think we shouls apply the current patch as-is, and leave the
> outlined follow-up patch as material for the tests-init branch.
Thinking again, it's better to get it right the first time.
Let's drop the previous patch.

Regards,
   Stefano

From fcf04d6920da7edfe6d6b8756c6fefce9dc4a004 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Fri, 3 Sep 2010 01:42:09 +0200
Subject: [PATCH] Testsuite: use $SHELL to run tests which are shell scripts.

* tests/Makefile.am (TEST_LOG_COMPILER): Define to use the
configure-time $SHELL to run the tests.
(TEST_EXTENSIONS): Defined to `.test', for clarity.
* tests/defs.in: Add code to re-execute by default the test
script using configure-time $SHELL.
* tests/README (Supported shells): Updated.
(Getting details from failures): Dont' tell that tests are
run by /bin/sh by default.
---
 ChangeLog         |   12 ++++++++++++
 tests/Makefile.am |    6 ++++++
 tests/Makefile.in |    6 +++++-
 tests/README      |   20 ++++++++++++++------
 tests/defs.in     |   31 ++++++++++++++++++++++++++++++-
 5 files changed, 67 insertions(+), 8 deletions(-)

diff --git a/ChangeLog b/ChangeLog
index 825589d..41c2464 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,15 @@
+2010-09-03  Stefano Lattarini  <address@hidden>
+
+       Testsuite: use $SHELL to run tests which are shell scripts.
+       * tests/Makefile.am (TEST_LOG_COMPILER): Define to use the
+       configure-time $SHELL to run the tests.
+       (TEST_EXTENSIONS): Defined to `.test', for clarity.
+       * tests/defs.in: Add code to re-execute by default the test
+       script using configure-time $SHELL.
+       * tests/README (Supported shells): Updated.
+       (Getting details from failures): Dont' tell that tests are
+       run by /bin/sh by default.
+
 2010-09-02  Peter Rosin  <address@hidden>
 
        Make ar-lib support backslashed files in archives.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index d86b03b..2fe6b87 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -16,6 +16,12 @@
 # You should have received a copy of the GNU General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+TEST_EXTENSIONS = .test
+# Run the tests with the shell detected at configure time.  Also, tell
+# them not to needlessly re-execute themselves with that shell, since
+# we know they are already running under it in our setup.
+TEST_LOG_COMPILER = AM_TESTS_REEXEC=no $(SHELL)
+
 XFAIL_TESTS =                                  \
 all.test                                       \
 auxdir2.test                                   \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 77ec888..fe48ac6 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -152,7 +152,6 @@ RECHECK_LOGS = $(TEST_LOGS)
 AM_RECURSIVE_TARGETS = check check-html recheck recheck-html
 TEST_SUITE_LOG = test-suite.log
 TEST_SUITE_HTML = $(TEST_SUITE_LOG:.log=.html)
-TEST_EXTENSIONS = .test
 am__test_logs1 = $(TESTS:=.log)
 TEST_LOGS = $(am__test_logs1:.test.log=.log)
 TEST_LOG_COMPILE = $(TEST_LOG_COMPILER) $(AM_TEST_LOG_FLAGS) \
@@ -250,6 +249,11 @@ target_alias = @target_alias@
 top_build_prefix = @top_build_prefix@
 top_builddir = @top_builddir@
 top_srcdir = @top_srcdir@
+TEST_EXTENSIONS = .test
+# Run the tests with the shell detected at configure time.  Also, tell
+# them not to needlessly re-execute themselves with that shell, since
+# we know they are already running under it in our setup.
+TEST_LOG_COMPILER = AM_TESTS_REEXEC=no $(SHELL)
 XFAIL_TESTS = \
 all.test                                       \
 auxdir2.test                                   \
diff --git a/tests/README b/tests/README
index 7037591..dbe653e 100644
--- a/tests/README
+++ b/tests/README
@@ -51,10 +51,10 @@ Interpretation
 Getting details from failures
 -----------------------------
 
-  Each test is a shell script, and by default is run by /bin/sh.
-  In a non-VPATH build you can run them directly, they will be verbose.
-  By default, verbose output of a test foo.test is retained in the log
-  file foo.log.  A summary log is created in the file test-suite.log.
+  Each test is a shell script.  In a non-VPATH build you can run the
+  tests directly, they will be verbose.  By default, verbose output of
+  a test foo.test is retained in the log file foo.log.  A summary log
+  is created in the file test-suite.log.
 
   You can limit the set of files using the TESTS variable, and enable
   detailed test output at the end of the test run with the VERBOSE
@@ -66,6 +66,15 @@ Getting details from failures
 Supported shells
 ----------------
 
+  By default, the tests are run by the $SHELL detected at configure
+  time.  They also takes care to re-execute themselves with that shell,
+  unless told not to.  So, to run the tests with a different shell, say
+  `/path/to/sh', the user must resort to something like:
+    $ AM_TESTS_REEXEC=no /path/to/sh ./foo.test
+  to run a test directly, and someting like:
+    $ AM_TESTS_REEXEC=no make check TEST_LOG_COMPILER=/path/to/sh
+  to run the test(s) through the makefile test driver.
+
   The test scripts are written with portability in mind, so that they
   should run with any decent Bourne-compatible shell.
 
@@ -75,8 +84,7 @@ Supported shells
   no easy workaround.  Thus, if you want to run a test script, say
   foo.test, with Zsh, you *can't* simply do `zsh foo.test', but you
   *must* resort to:
-    zsh -o no_function_argzero foo.test
-
+    AM_TESTS_REEXEC=no zsh -o no_function_argzero foo.test
   Note that this problem does not occur if zsh is executed through a
   symlink with a basename of `sh', since in that case it starts
   in Bourne compatibility mode.  So you should be perfectly safe when
diff --git a/tests/defs.in b/tests/defs.in
index b5cc99e..fd7e06b 100644
--- a/tests/defs.in
+++ b/tests/defs.in
@@ -55,11 +55,40 @@ test -f "$srcdir/defs.in" || {
    exit 1
 }
 
-me=`echo "$0" | sed -e 's,.*[\\/],,;s/\.test$//'`
+# Ensure we can find ourselves.
+test -f "$0" || {
+   echo "test unable to find itself: $0" 1>&2
+   exit 1
+}
 
 # Make sure we override the user shell.
 SHELL='@SHELL@'
 export SHELL
+
+me=`echo "$0" | sed -e 's,.*[\\/],,;s/\.test$//'`
+
+# Make sure we run with the shell detected at configure time (unless
+# the user forbids it).
+case ${AM_TESTS_REEXEC-'yes'} in
+  1|[yY]|[yY]es)
+    AM_TESTS_REEXEC=no; export AM_TESTS_REEXEC
+    # Cannot simply do `opts=$-', since the content of $- is not
+    # portable among different shells.  So try to propagate only
+    # the portable and interesting options.
+    case $- in
+      *x*v*|*v*x) opts=-vx;;
+      *v*) opts=-v;;
+      *x*) opts=-x;;
+      *) opts=;;
+    esac
+    echo $me: exec $SHELL $opts "$0" "$*"
+    exec $SHELL $opts "$0" ${1+"$@"}
+    # NOTREACHED
+    echo "$me: failed to re-execute with $SHELL" >&2
+    exit 99
+    ;;
+esac
+
 # User can override various tools used.
 test -z "$PERL" && PERL='@PERL@'
 test -z "$MAKE" && MAKE=make
-- 
1.7.1