FYI: tests/README

[Alexandre Duret-Lutz]

I'm checking this in, on HEAD and branch-1-7.

2003-01-19  Alexandre Duret-Lutz  <address@hidden>

        * tests/README: New file.

Index: tests/README
===================================================================
RCS file: tests/README
diff -N tests/README
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ tests/README        19 Jan 2003 15:34:18 -0000
@@ -0,0 +1,130 @@
+                           The Automake test suite
+
+
+User interface
+==============
+
+
+Running all tests
+-----------------
+
+  make check
+
+
+Interpretation
+--------------
+
+  Successes:
+    PASS  - success
+    XFAIL - expected failure
+
+  Failures:
+    FAIL  - failure
+    XPASS - unexpected success
+
+  Other:
+    SKIP  - skipped tests (third party tools not available)
+
+
+Getting details from failures
+-----------------------------
+
+  Each test is a script.  In a non-VPATH build you can run them
+  directly, they will be verbose.
+
+  Otherwise, you can invoke make as follows, just replace the list
+  of tests by those you want to check.
+
+    env VERBOSE=x TESTS='first.test second.test ...' make -e check
+
+
+Reporting failures
+------------------
+
+  Send verbose output of failing tests to <address@hidden>,
+  along with the usual version numbers (which Automake, which
+  Autoconf, which operating system, which make version, which shell,
+  etc.)
+
+
+
+Writing test cases
+==================
+
+
+Do
+--
+
+  If you plan to fix a bug, write the test case first.  This way you'll
+  make sure the test catches the bug, and that is succeeds once you have
+  fixed the bug.
+
+  Add a copyright/license paragraph.
+
+  Explain what the test does.
+
+  Cite the PR number (if any), and the original reporter (if any), so
+  we can find or ask for information if needed.
+
+  Use `required=...' for required tools.
+
+  Include ./defs (see other tests).
+
+  Use `set -e' to catch failures you might not have thought of.
+
+  ./defs sets a skeleton configure.in.  If possible, append to this
+  file.  In some cases you'll have to overwrite it, but this should
+  be the exception.  Note that configure.in registers Makefile.in
+  but do not output anything by default.  If you need ./configure
+  to create Makefile, append AC_OUTPUT to configure.in.
+
+  Use $ACLOCAL, $AUTOMAKE, $AUTOCONF, $AUTOUPDATE, $AUTOHEADER,
+  $PERL, $MAKE, $EGREP, and $FGREP, instead of the corresponding
+  commands.
+
+  Use $sleep when you have to make sure that some file is newer
+  than another.
+
+  Use `cat' or `grep' to display (part of) files that may be
+  interesting for debugging, so that when a user send a verbose
+  output we don't have to ask him for more details.
+
+  It's more important to make sure that a feature works, than
+  make sure that Automake's output looks correct.  It might look
+  correct and still fails to work.  In other words, prefer
+  running `make' over grepping `Makefile.in' (or do both).
+
+  If you run $AUTOMAKE or $AUTOCONF several times in the same test
+  and change `configure.in' by the meantime, do
+    rm -rf autom4te.cache
+  before the following runs.  On fast machines the new `configure.in'
+  could otherwise have the same timestamp as the old `autom4te.cache'.
+
+  Before commit: make sure the test is executable, add the tests to
+  TESTS in Makefile.am, add it to XFAIL_TESTS in addition if needed,
+  write a ChangeLog entry, send the diff to <address@hidden>.
+
+
+Do not
+------
+
+  Do not test an Automake error with `$AUTOMAKE && exit 1', or in three
+  years we'll discover that this test failed for some other bogus reason.
+  This happened many times.  Better use something like
+     $AUTOMAKE 2>stderr && exit 1
+     cat stderr
+     grep 'expected diagnostic' stderr
+  (Note this doesn't prevent the test from failing for another
+  reason, but at least it makes sure the original error is still
+  here.)
+
+  Do not override Makefile variables using make arguments, as in
+    $MAKE ANSI2KNR=./ansi2knr U=_ all
+  this is not portable for recursive targets (targets that
+  call a sub-make may not pass `ANSI2KNR=./ansi2knr U=_' along).
+  Use the following instead.
+    ANSI2KNR=./ansi2knr U=_ $MAKE -e all
+
+  Do not send a test case without signing a copyright disclaimer.
+  See http://sources.redhat.com/automake/contribute.html or
+  ask <address@hidden> for details.

-- 
Alexandre Duret-Lutz