automake-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 05] Overhaul of tests/README.


From: Ralf Wildenhues
Subject: Re: [PATCH 05] Overhaul of tests/README.
Date: Sat, 14 Mar 2009 11:53:24 +0100
User-agent: Mutt/1.5.18 (2008-05-17)

    Overhaul of tests/README.
    
    * tests/README: General overhaul.  Mention parallel test suite
    execution, the test-suite.log file; document `Exit' vs. `exit',
    `defs' vs. `defs-p', the naming of `parallel-tests' tests.
    * README: Simplify, point to tests/README.

diff --git a/README b/README
index bb42a7f..0e23237 100644
--- a/README
+++ b/README
@@ -24,13 +24,7 @@ Automake has a test suite.  Use
        make check
 
 to run it.  Capture its output in case of failing tests.  For more
-information, you can influence testing with the variables VERBOSE,
-MAKE, TESTS, keep_testdirs.  So for example, to run tests/check.test
-verbosely using gmake and to inspect the test directory afterwards,
-enter the tests directory and use
-
-       env VERBOSE=yes TESTS=check.test keep_testdirs=yes MAKE=gmake \
-         gmake -e check
+information, please see the file tests/README.
 
 Automake has a page on the web.  See:
 
diff --git a/tests/README b/tests/README
index 4b63059..c4ff407 100644
--- a/tests/README
+++ b/tests/README
@@ -10,6 +10,9 @@ Running all tests
 
   make check
 
+  You can use `-jN' for faster completion (it even helps on a
+  uniprocessor system, due to unavoidable sleep delays, as
+  noted below).
 
 Interpretation
 --------------
@@ -30,10 +33,13 @@ Getting details from failures
 -----------------------------
 
   Each test is a script.  In a non-VPATH build you can run them
-  directly, they will be verbose.
+  directly, they will be verbose.  By default, verbose output of
+  a test foo.test is retained in the log file foo.log.  A summary
+  log is created in the file test-suite.log.
 
-  Otherwise, you can invoke make as follows, just replace the list
-  of tests by those you want to check.
+  You can limit the set of files using the TESTS variable, and
+  enable detailed test output at the end of the test run with the
+  VERBOSE variable:
 
     env VERBOSE=x TESTS='first.test second.test ...' make -e check
 
@@ -41,10 +47,10 @@ Getting details from failures
 Reporting failures
 ------------------
 
-  Send verbose output of failing tests to <address@hidden>,
-  along with the usual version numbers (which Automake, which
-  Autoconf, which operating system, which make version, which shell,
-  etc.)
+  Send verbose output, i.e., the contents of test-suite.log, of failing
+  tests to <address@hidden>, along with the usual version numbers
+  (which Automake, which Autoconf, which operating system, which make
+  version, which shell, etc.)
 
 
 
@@ -68,7 +74,10 @@ Do
 
   Use `required=...' for required tools.
 
-  Include ./defs (see other tests).
+  Include ./defs (see other tests) for normal tests, ./defs-p for tests
+  that use the `parallel-tests' option.  For the latter, use a name that
+  ends in `-p.test' and does not clash with any generated tests in the
+  suite.
 
   Use `set -e' to catch failures you might not have thought of.
 
@@ -87,11 +96,16 @@ Do
 
   Use `cat' or `grep' to display (part of) files that may be
   interesting for debugging, so that when a user send a verbose
-  output we don't have to ask him for more details.
+  output we don't have to ask him for more details.  Display stderr
+  output on the stderr file descriptor.  If some redirected command
+  is likely to fail, and `set -e' is in effect, display its output
+  even in the failure case, before exiting.
+
+  Use `Exit' rather than `exit' to abort a test.
 
   It's more important to make sure that a feature works, than
   make sure that Automake's output looks correct.  It might look
-  correct and still fails to work.  In other words, prefer
+  correct and still fail to work.  In other words, prefer
   running `make' over grepping `Makefile.in' (or do both).
 
   If you run $AUTOMAKE or $AUTOCONF several times in the same test
@@ -99,6 +113,7 @@ Do
     rm -rf autom4te.cache
   before the following runs.  On fast machines the new `configure.in'
   could otherwise have the same timestamp as the old `autom4te.cache'.
+  Alternatively, use `--force' for subsequent runs of the tools.
 
   Use filenames with two consecutive spaces when testing that some
   code preserves filenames with spaces.  This will catch errors like
@@ -112,7 +127,7 @@ Do
 Do not
 ------
 
-  Do not test an Automake error with `$AUTOMAKE && exit 1', or in three
+  Do not test an Automake error with `$AUTOMAKE && Exit 1', or in three
   years we'll discover that this test failed for some other bogus reason.
   This happened many times.  Better use something like
      AUTOMAKE_fails




reply via email to

[Prev in Thread] Current Thread [Next in Thread]