[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
FYI: tests/README
From: |
Alexandre Duret-Lutz |
Subject: |
FYI: tests/README |
Date: |
Sun, 19 Jan 2003 16:36:25 +0100 |
User-agent: |
Gnus/5.090008 (Oort Gnus v0.08) Emacs/21.2 (i386-pc-linux-gnu) |
I'm checking this in, on HEAD and branch-1-7.
2003-01-19 Alexandre Duret-Lutz <address@hidden>
* tests/README: New file.
Index: tests/README
===================================================================
RCS file: tests/README
diff -N tests/README
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ tests/README 19 Jan 2003 15:34:18 -0000
@@ -0,0 +1,130 @@
+ The Automake test suite
+
+
+User interface
+==============
+
+
+Running all tests
+-----------------
+
+ make check
+
+
+Interpretation
+--------------
+
+ Successes:
+ PASS - success
+ XFAIL - expected failure
+
+ Failures:
+ FAIL - failure
+ XPASS - unexpected success
+
+ Other:
+ SKIP - skipped tests (third party tools not available)
+
+
+Getting details from failures
+-----------------------------
+
+ Each test is a script. In a non-VPATH build you can run them
+ directly, they will be verbose.
+
+ Otherwise, you can invoke make as follows, just replace the list
+ of tests by those you want to check.
+
+ env VERBOSE=x TESTS='first.test second.test ...' make -e check
+
+
+Reporting failures
+------------------
+
+ Send verbose output of failing tests to <address@hidden>,
+ along with the usual version numbers (which Automake, which
+ Autoconf, which operating system, which make version, which shell,
+ etc.)
+
+
+
+Writing test cases
+==================
+
+
+Do
+--
+
+ If you plan to fix a bug, write the test case first. This way you'll
+ make sure the test catches the bug, and that is succeeds once you have
+ fixed the bug.
+
+ Add a copyright/license paragraph.
+
+ Explain what the test does.
+
+ Cite the PR number (if any), and the original reporter (if any), so
+ we can find or ask for information if needed.
+
+ Use `required=...' for required tools.
+
+ Include ./defs (see other tests).
+
+ Use `set -e' to catch failures you might not have thought of.
+
+ ./defs sets a skeleton configure.in. If possible, append to this
+ file. In some cases you'll have to overwrite it, but this should
+ be the exception. Note that configure.in registers Makefile.in
+ but do not output anything by default. If you need ./configure
+ to create Makefile, append AC_OUTPUT to configure.in.
+
+ Use $ACLOCAL, $AUTOMAKE, $AUTOCONF, $AUTOUPDATE, $AUTOHEADER,
+ $PERL, $MAKE, $EGREP, and $FGREP, instead of the corresponding
+ commands.
+
+ Use $sleep when you have to make sure that some file is newer
+ than another.
+
+ Use `cat' or `grep' to display (part of) files that may be
+ interesting for debugging, so that when a user send a verbose
+ output we don't have to ask him for more details.
+
+ It's more important to make sure that a feature works, than
+ make sure that Automake's output looks correct. It might look
+ correct and still fails to work. In other words, prefer
+ running `make' over grepping `Makefile.in' (or do both).
+
+ If you run $AUTOMAKE or $AUTOCONF several times in the same test
+ and change `configure.in' by the meantime, do
+ rm -rf autom4te.cache
+ before the following runs. On fast machines the new `configure.in'
+ could otherwise have the same timestamp as the old `autom4te.cache'.
+
+ Before commit: make sure the test is executable, add the tests to
+ TESTS in Makefile.am, add it to XFAIL_TESTS in addition if needed,
+ write a ChangeLog entry, send the diff to <address@hidden>.
+
+
+Do not
+------
+
+ Do not test an Automake error with `$AUTOMAKE && exit 1', or in three
+ years we'll discover that this test failed for some other bogus reason.
+ This happened many times. Better use something like
+ $AUTOMAKE 2>stderr && exit 1
+ cat stderr
+ grep 'expected diagnostic' stderr
+ (Note this doesn't prevent the test from failing for another
+ reason, but at least it makes sure the original error is still
+ here.)
+
+ Do not override Makefile variables using make arguments, as in
+ $MAKE ANSI2KNR=./ansi2knr U=_ all
+ this is not portable for recursive targets (targets that
+ call a sub-make may not pass `ANSI2KNR=./ansi2knr U=_' along).
+ Use the following instead.
+ ANSI2KNR=./ansi2knr U=_ $MAKE -e all
+
+ Do not send a test case without signing a copyright disclaimer.
+ See http://sources.redhat.com/automake/contribute.html or
+ ask <address@hidden> for details.
--
Alexandre Duret-Lutz
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- FYI: tests/README,
Alexandre Duret-Lutz <=