[PATCH] {maint} tests: better handling of gettext and libtool requirements

[Stefano Lattarini]

This change fixes automake bug#9807.

Before this change, the automake testsuite only looked for the
`.m4' files containing libtool and gettext macros definitions in
the directory `${prefix}/share/aclocal' (and in the directories
specified by the `dirlist' file in there, if any), where ${prefix}
was the configure-time automake installation prefix (defaulting
to `/usr/local').

This approach had various shortcomings and disadvantages.  Let's
briefly describe the three major ones.

First, on most GNU/Linux systems, a libtool or gettext installed
from distro-provided packages (e.g., by dpkg on Debian/Ubuntu, or
by rmp on RedHat/Fedora) would have `/usr', not `/usr/local', as
its ${prefix}; so, trying to run the automake testsuite with a
simple "./configure && make && make check" would have failed to
execute the libtool and gettext tests on most GNU/Linux distros.
It's true that it was quite easy to work around this issue, by
creating a proper `/usr/local/share/aclocal/dirlist' file with
an entry pointing to `/usr/share/aclocal' (a workaround in fact
used by most automake developers); but the typical user wasn't
aware of the necessity of this trick, so the libtool and gettext
tests was usually skipped on testsuite runs "in the wild", thus
needlessly reducing coverage.

Second, the older testsuite behaviour made more difficult for
the developers to run the testsuite with non-default libtool or
gettext.  For example, assume the developer is working on a system
that has a default libtool version 1.5 installed in the /usr/local
hierarchy; to improve coverage, the developer installs also a more
modern libtool version, say 2.4, in its home directory, let's say
in ~/libtool-2.4; he then tries to run the automake testsuite with
this more modern libtool by doing an (apparently) simple:
  $ PATH=$HOME/libtool-2.4:$PATH make check
But the automake testsuite would still look for libtool macros in
/usr/local/share/aclocal, not in ~/libtool-2.4/share/aclocal, so
the wrong version of the macros would be picked up, and the tests
would either fail spuriously or (which would be worse) pass without
truly covering the libtool version the developers was thinking to
be testing with.
Worse again, the automake testsuite would *unconditionally* look
for libtool macros in /usr/local/share/aclocal, so even something
like:
  $ export ACLOCAL_PATH=$HOME/libtool-2.4/share/aclocal
  $ PATH=$HOME/libtool-2.4:$PATH make check
wouldn't work.

Third and last, during a "make distcheck", automake is configured
with a ${prefix} pointing to a proper subdirectory of the build
directory (usually `pwd`/_inst), which gets created on-the-fly;
in this case, with the old approach, the automake testsuite never
found the libtool and gettext macro files, ans so the libtool and
gettext tests was *always* skipped in a "make distcheck".

* tests/libtool-macros.test: New helper test, looking (with the
help of the `libtoolize' script) for libtool macro files required
by most libtool tests, and making them easily accessible.
* tests/gettext-macros.test: New helper test, looking (with the
help of the `libtoolize' script) for libtool macro files required
by most libtool tests, and making them easily accessible.
* tests/defs.in: Update to make it rely on the results and setups
of `libtool-macros.test' and `gettext-macros.test'.
* tests/Makefile.am: Declare dependency of all the logs of libtool
tests from `libtool-macros.log', and all the logs of gettext tests
from `gettext-macros.log'.
---
 ChangeLog                 |   71 +++++++++++++++++++++++++++++++++++++++
 tests/Makefile.am         |   62 ++++++++++++++++++++++++++++++++++
 tests/Makefile.in         |   62 ++++++++++++++++++++++++++++++++++
 tests/defs.in             |   43 +-----------------------
 tests/gettext-macros.test |   80 +++++++++++++++++++++++++++++++++++++++++++++
 tests/libtool-macros.test |   62 ++++++++++++++++++++++++++++++++++
 6 files changed, 339 insertions(+), 41 deletions(-)
 create mode 100755 tests/gettext-macros.test
 create mode 100755 tests/libtool-macros.test

diff --git a/ChangeLog b/ChangeLog
index 826a4f7..6527e31 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,74 @@
+2011-12-14  Stefano Lattarini  <address@hidden>
+
+       tests: better handling of gettext and libtool requirements
+
+       This change fixes automake bug#9807.
+
+       Before this change, the automake testsuite only looked for the
+       `.m4' files containing libtool and gettext macros definitions in
+       the directory `${prefix}/share/aclocal' (and in the directories
+       specified by the `dirlist' file in there, if any), where ${prefix}
+       was the configure-time automake installation prefix (defaulting
+       to `/usr/local').
+
+       This approach had various shortcomings and disadvantages.  Let's
+       briefly describe the three major ones.
+
+       First, on most GNU/Linux systems, a libtool or gettext installed
+       from distro-provided packages (e.g., by dpkg on Debian/Ubuntu, or
+       by rmp on RedHat/Fedora) would have `/usr', not `/usr/local', as
+       its ${prefix}; so, trying to run the automake testsuite with a
+       simple "./configure && make && make check" would have failed to
+       execute the libtool and gettext tests on most GNU/Linux distros.
+       It's true that it was quite easy to work around this issue, by
+       creating a proper `/usr/local/share/aclocal/dirlist' file with
+       an entry pointing to `/usr/share/aclocal' (a workaround in fact
+       used by most automake developers); but the typical user wasn't
+       aware of the necessity of this trick, so the libtool and gettext
+       tests was usually skipped on testsuite runs "in the wild", thus
+       needlessly reducing coverage.
+
+       Second, the older testsuite behaviour made more difficult for
+       the developers to run the testsuite with non-default libtool or
+       gettext.  For example, assume the developer is working on a system
+       that has a default libtool version 1.5 installed in the /usr/local
+       hierarchy; to improve coverage, the developer installs also a more
+       modern libtool version, say 2.4, in its home directory, let's say
+       in ~/libtool-2.4; he then tries to run the automake testsuite with
+       this more modern libtool by doing an (apparently) simple:
+         $ PATH=$HOME/libtool-2.4:$PATH make check
+       But the automake testsuite would still look for libtool macros in
+       /usr/local/share/aclocal, not in ~/libtool-2.4/share/aclocal, so
+       the wrong version of the macros would be picked up, and the tests
+       would either fail spuriously or (which would be worse) pass without
+       truly covering the libtool version the developers was thinking to
+       be testing with.
+       Worse again, the automake testsuite would *unconditionally* look
+       for libtool macros in /usr/local/share/aclocal, so even something
+       like:
+         $ export ACLOCAL_PATH=$HOME/libtool-2.4/share/aclocal
+         $ PATH=$HOME/libtool-2.4:$PATH make check
+       wouldn't work.
+
+       Third and last, during a "make distcheck", automake is configured
+       with a ${prefix} pointing to a proper subdirectory of the build
+       directory (usually `pwd`/_inst), which gets created on-the-fly;
+       in this case, with the old approach, the automake testsuite never
+       found the libtool and gettext macro files, ans so the libtool and
+       gettext tests was *always* skipped in a "make distcheck".
+
+       * tests/libtool-macros.test: New helper test, looking (with the
+       help of the `libtoolize' script) for libtool macro files required
+       by most libtool tests, and making them easily accessible.
+       * tests/gettext-macros.test: New helper test, looking (with the
+       help of the `libtoolize' script) for libtool macro files required
+       by most libtool tests, and making them easily accessible.
+       * tests/defs.in: Update to make it rely on the results and setups
+       of `libtool-macros.test' and `gettext-macros.test'.
+       * tests/Makefile.am: Declare dependency of all the logs of libtool
+       tests from `libtool-macros.log', and all the logs of gettext tests
+       from `gettext-macros.log'.
+
 2011-12-10  Stefano Lattarini  <address@hidden>
 
        release: don't run "make distcheck" automatically
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 831906b..d6a15f7 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -924,6 +924,68 @@ yflags.test \
 yflags2.test \
 $(parallel_tests)
 
+# FIXME: make these automatically computed once we are merged into
+# FIXME: the `testsuite-work' branch.
+depcomp4.log: libtool-macros.log
+depcomp7.log: libtool-macros.log
+depcomp8b.log: libtool-macros.log
+fort5.log: libtool-macros.log
+instdir-ltlib.log: libtool-macros.log
+instfail-libtool.log: libtool-macros.log
+ldadd.log: libtool-macros.log
+ldflags.log: libtool-macros.log
+libobj13.log: libtool-macros.log
+libtoo10.log: libtool-macros.log
+libtoo11.log: libtool-macros.log
+libtool.log: libtool-macros.log
+libtool2.log: libtool-macros.log
+libtool3.log: libtool-macros.log
+libtool5.log: libtool-macros.log
+libtool6.log: libtool-macros.log
+libtool7.log: libtool-macros.log
+libtool8.log: libtool-macros.log
+libtool9.log: libtool-macros.log
+listval.log: libtool-macros.log
+ltcond.log: libtool-macros.log
+ltcond2.log: libtool-macros.log
+ltconv.log: libtool-macros.log
+ltdeps.log: libtool-macros.log
+ltinit.log: libtool-macros.log
+ltinstloc.log: libtool-macros.log
+ltlibobjs.log: libtool-macros.log
+ltlibsrc.log: libtool-macros.log
+ltorder.log: libtool-macros.log
+nobase-libtool.log: libtool-macros.log
+pr211.log: libtool-macros.log
+pr300-ltlib.log: libtool-macros.log
+pr307.log: libtool-macros.log
+pr401b.log: libtool-macros.log
+pr72.log: libtool-macros.log
+reqd2.log: libtool-macros.log
+silent3.log: libtool-macros.log
+silent4.log: libtool-macros.log
+silent9.log: libtool-macros.log
+stdlib2.log: libtool-macros.log
+strip3.log: libtool-macros.log
+subobj9.log: libtool-macros.log
+suffix10.log: libtool-macros.log
+suffix2.log: libtool-macros.log
+suffix5.log: libtool-macros.log
+suffix8.log: libtool-macros.log
+vala.log: libtool-macros.log
+vala1.log: libtool-macros.log
+vala2.log: libtool-macros.log
+vala3.log: libtool-macros.log
+vala4.log: libtool-macros.log
+vala5.log: libtool-macros.log
+
+# FIXME: make these automatically computed once we are merged into
+# FIXME: the `testsuite-work' branch.
+gettext.log: gettext-macros.log
+gettext2.log: gettext-macros.log
+gettext3.log: gettext-macros.log
+subcond.log: gettext-macros.log
+
 EXTRA_DIST = ChangeLog-old gen-parallel-tests $(TESTS)
 
 distcheck-missing-m4.log distcheck-outdated-m4.log: distcheck-hook-m4.am
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 3ad0146..adbd77e 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1547,6 +1547,68 @@ $(parallel_tests): $(parallel_tests:-p.test=.test) 
Makefile.am
          < $(srcdir)/$$input >$@
        chmod a+rx $@
 
+# FIXME: make these automatically computed once we are merged into
+# FIXME: the `testsuite-work' branch.
+depcomp4.log: libtool-macros.log
+depcomp7.log: libtool-macros.log
+depcomp8b.log: libtool-macros.log
+fort5.log: libtool-macros.log
+instdir-ltlib.log: libtool-macros.log
+instfail-libtool.log: libtool-macros.log
+ldadd.log: libtool-macros.log
+ldflags.log: libtool-macros.log
+libobj13.log: libtool-macros.log
+libtoo10.log: libtool-macros.log
+libtoo11.log: libtool-macros.log
+libtool.log: libtool-macros.log
+libtool2.log: libtool-macros.log
+libtool3.log: libtool-macros.log
+libtool5.log: libtool-macros.log
+libtool6.log: libtool-macros.log
+libtool7.log: libtool-macros.log
+libtool8.log: libtool-macros.log
+libtool9.log: libtool-macros.log
+listval.log: libtool-macros.log
+ltcond.log: libtool-macros.log
+ltcond2.log: libtool-macros.log
+ltconv.log: libtool-macros.log
+ltdeps.log: libtool-macros.log
+ltinit.log: libtool-macros.log
+ltinstloc.log: libtool-macros.log
+ltlibobjs.log: libtool-macros.log
+ltlibsrc.log: libtool-macros.log
+ltorder.log: libtool-macros.log
+nobase-libtool.log: libtool-macros.log
+pr211.log: libtool-macros.log
+pr300-ltlib.log: libtool-macros.log
+pr307.log: libtool-macros.log
+pr401b.log: libtool-macros.log
+pr72.log: libtool-macros.log
+reqd2.log: libtool-macros.log
+silent3.log: libtool-macros.log
+silent4.log: libtool-macros.log
+silent9.log: libtool-macros.log
+stdlib2.log: libtool-macros.log
+strip3.log: libtool-macros.log
+subobj9.log: libtool-macros.log
+suffix10.log: libtool-macros.log
+suffix2.log: libtool-macros.log
+suffix5.log: libtool-macros.log
+suffix8.log: libtool-macros.log
+vala.log: libtool-macros.log
+vala1.log: libtool-macros.log
+vala2.log: libtool-macros.log
+vala3.log: libtool-macros.log
+vala4.log: libtool-macros.log
+vala5.log: libtool-macros.log
+
+# FIXME: make these automatically computed once we are merged into
+# FIXME: the `testsuite-work' branch.
+gettext.log: gettext-macros.log
+gettext2.log: gettext-macros.log
+gettext3.log: gettext-macros.log
+subcond.log: gettext-macros.log
+
 distcheck-missing-m4.log distcheck-outdated-m4.log: distcheck-hook-m4.am
 
 clean-local: clean-local-check
diff --git a/tests/defs.in b/tests/defs.in
index f24d2ad..a938675 100644
--- a/tests/defs.in
+++ b/tests/defs.in
@@ -443,47 +443,8 @@ unset VERBOSE
 echo "=== Running test $0"
 
 # We might need extra macros, e.g., from Libtool or Gettext.
-# Find them on the system.
-# Use `-I $srcdir/../m4' in addition to `--acdir=$srcdir/../m4', because the
-# other `-I' directories added for libtool and gettext might contain
-# files from an old version of Automake that we don't want to use.
-# Use `-Wno-syntax' because we do not want our test suite to fail because
-# some third-party .m4 file is underquoted.
-case $required in
-  *libtool* | *gettext* )
-    aclocaldir='@prefix@/share/aclocal'
-    extra_includes=""
-    if test -f $aclocaldir/dirlist; then
-       extra_includes=`
-       <$aclocaldir/dirlist \
-       sed  's/#.*//;s/[        ][      ]*$//g' \
-       | while read dir; do test ! -d "$dir" || echo "-I $dir"; done`
-    else :; fi
-
-    libtool_found=no
-    gettext_found=no
-    for d in $extra_includes $aclocaldir ; do
-       test "x$d" != x-I || continue
-       if test -f "$d/libtool.m4"; then
-         libtool_found=yes
-       fi
-       if test -f "$d/gettext.m4"; then
-         gettext_found=yes
-       fi
-    done
-    case $required in
-      *libtool* ) test $libtool_found = yes || Exit 77 ;;
-      *gettext* ) test $gettext_found = yes || Exit 77 ;;
-    esac
-    # Libtool cannot cope with spaces in the build tree.  Our testsuite setup
-    # cannot cope with spaces in the source tree name for Libtool and gettext
-    # tests.
-    case $srcdir,`pwd` in
-      *\ * | *\        *) Exit 77 ;;
-    esac
-    ACLOCAL="$ACLOCAL -Wno-syntax -I $srcdir/../m4 $extra_includes -I 
$aclocaldir"
-    ;;
-esac
+case " $required " in *\ libtool*) . ../libtool-macros.dir/get.sh;; esac
+case " $required " in *\ gettext*) . ../gettext-macros.dir/get.sh;; esac
 
 testaclocaldir='@abs_top_srcdir@/m4'
 
diff --git a/tests/gettext-macros.test b/tests/gettext-macros.test
new file mode 100755
index 0000000..7fe1274
--- /dev/null
+++ b/tests/gettext-macros.test
@@ -0,0 +1,80 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Try to find the gettext `.m4' files and make them easily accessed
+# to the test cases requiring them.
+# See also automake bug#9807.
+
+. ./defs || Exit 1
+
+echo "# Automatically generated by $me." > get.sh
+echo : >> get.sh
+
+# The `gettextize' and `autopoint' scripts will look into Makefile.am.
+echo ACLOCAL_AMFLAGS = -I m4 > Makefile.am
+
+# Required by autopoint.
+echo 'AM_GNU_GETTEXT' > configure.in
+# Likewise; and older version specified here *won't* work!
+echo 'AM_GNU_GETTEXT_VERSION([0.10.35])' >> configure.in
+
+# Prefer autopoint to gettextize, since the more modern versions of the
+# latter might unconditionally require user interaction to complete;
+# yes, this means confirmation from /dev/tty (!) -- see:
+#  <http://lists.gnu.org/archive/html/bug-gettext/2011-12/msg00000.html>
+# Since this "forced interaction" behaviour of gettextize wasn't present
+# before the introduction of autopoint, we should be able to safely
+# fall back to calling gettextize non-interactively if autopoint is not
+# present.
+if autopoint --version; then
+  am_gettextize_command=autopoint
+else
+  am_gettextize_command=gettextize
+fi
+
+if $am_gettextize_command --force && test -f m4/gettext.m4; then
+  unindent >> get.sh <<END
+    ACLOCAL_PATH="`pwd`/m4":\$ACLOCAL_PATH
+    export ACLOCAL_PATH
+END
+else
+  # Older versions of gettext might not have a gettextize program
+  # available, but this doesn't mean the user hasn't made the gettext
+  # macros available, e.g., by properly setting ACLOCAL_PATH.
+  rm -rf m4
+  mkdir m4
+  # See below for an explanation about the use the of `-Wno-syntax'.
+  if $ACLOCAL -Wno-syntax -I m4 --install && test -f m4/gettext.m4; then
+    : # Gettext macros already accessible by default.
+  else
+    echo "skip_ \"couldn't find or get gettext macros\"" >> get.sh
+  fi
+fi
+
+. ./get.sh
+
+$ACLOCAL --force -I m4 || cat >> get.sh <<'END'
+# We need to use `-Wno-syntax', since we do not want our test suite
+# to fail merely because some third-party `.m4' file is underquoted.
+ACLOCAL="$ACLOCAL -Wno-syntax"
+END
+
+# The file gettextize or autopoint might have copied in the `m4'
+# subdirectory of the test directory are going to be needed by
+# other tests, so we must not remove the test directory.
+keep_testdirs=yes
+
+:
diff --git a/tests/libtool-macros.test b/tests/libtool-macros.test
new file mode 100755
index 0000000..31e5019
--- /dev/null
+++ b/tests/libtool-macros.test
@@ -0,0 +1,62 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Try to find the libtool `.m4' files and make them easily accessed
+# to the test cases requiring them.
+# See also automake bug#9807.
+
+. ./defs || Exit 1
+
+echo "# Automatically generated by $me." > get.sh
+echo : >> get.sh
+
+# The `libtoolize' script will look into Makefile.am.
+echo ACLOCAL_AMFLAGS = -I m4 > Makefile.am
+
+if libtoolize --copy --install && test -f m4/libtool.m4; then
+  unindent >> get.sh <<END
+    ACLOCAL_PATH="`pwd`/m4":\$ACLOCAL_PATH
+    export ACLOCAL_PATH
+END
+else
+  # Libtoolize from libtool < 2.0 didn't support the `--install' option,
+  # but this doesn't mean the user hasn't made the libtool macros
+  # available, e.g., by properly setting ACLOCAL_PATH.
+  rm -rf m4
+  mkdir m4
+  echo AC_PROG_LIBTOOL >> configure.in
+  # See below for an explanation about the use the of `-Wno-syntax'.
+  if $ACLOCAL -Wno-syntax -I m4 --install && test -f m4/libtool.m4; then
+    : # Libtool macros already accessible by default.
+  else
+    echo "skip_ \"couldn't find or get libtool macros\"" >> get.sh
+  fi
+fi
+
+. ./get.sh
+
+$ACLOCAL --force -I m4 || cat >> get.sh <<'END'
+# We need to use `-Wno-syntax', since we do not want our test suite
+# to fail merely because some third-party `.m4' file is underquoted.
+ACLOCAL="$ACLOCAL -Wno-syntax"
+END
+
+# The file libtoolize might have just copied in the `m4' subdirectory of
+# the test directory are going to be needed by other tests, so we must
+# not remove the test directory.
+keep_testdirs=yes
+
+:
-- 
1.7.2.3