[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[FYI 4/7] {tap-test-protocols} testsuite: update "user interface" descri
From: |
Stefano Lattarini |
Subject: |
[FYI 4/7] {tap-test-protocols} testsuite: update "user interface" description in tests/README |
Date: |
Sun, 7 Aug 2011 16:05:35 +0200 |
* tests/README (Subsection "Interpretation"): Mention the "ERROR"
outcome.
(Subsection "Getting details from failures"): Don't tell that the
test scripts can be run directly, this is not 100% true; anymore;
instead ...
(Subsection "About the tests"): ... use this new subsection to
explain how to run the tests (either simple or TAP-based) by hand.
(Subsection "Supported shells"): Extend to refer to TAP tests too.
---
ChangeLog | 12 ++++++++++++
tests/README | 37 +++++++++++++++++++++++++++++++------
2 files changed, 43 insertions(+), 6 deletions(-)
diff --git a/ChangeLog b/ChangeLog
index aa71382..a87953c 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,5 +1,17 @@
2011-08-07 Stefano Lattarini <address@hidden>
+ testsuite: update "user interface" description in tests/README
+ * tests/README (Subsection "Interpretation"): Mention the "ERROR"
+ outcome.
+ (Subsection "Getting details from failures"): Don't tell that the
+ test scripts can be run directly, this is not 100% true; anymore;
+ instead ...
+ (Subsection "About the tests"): ... use this new subsection to
+ explain how to run the tests (either simple or TAP-based) by hand.
+ (Subsection "Supported shells"): Extend to refer to TAP tests too.
+
+2011-08-07 Stefano Lattarini <address@hidden>
+
testsuite: TAP tests can have early plan now
* tests/defs: When using TAP, allow the user to specify the number
of tests through the `$planned' variable.
diff --git a/tests/README b/tests/README
index b3ff506..8b8da14 100644
--- a/tests/README
+++ b/tests/README
@@ -56,15 +56,15 @@ Interpretation
Other:
SKIP - skipped tests (third party tools not available)
+ ERROR - some unexpected error condition
Getting details from failures
-----------------------------
- Each test is a shell script. In a non-VPATH build you can run the
- tests directly, they will be verbose. By default, verbose output of
- a test foo.test is retained in the log file foo.log. A summary log
- is created in the file test-suite.log.
+ By default, verbose output of a test `foo.test' or `foo.tap' is
+ retained in the log file `foo.log'. A summary log is created in
+ the file `test-suite.log'.
You can limit the set of files using the TESTS variable, and enable
detailed test output at the end of the test run with the VERBOSE
@@ -72,6 +72,30 @@ Getting details from failures
env VERBOSE=x TESTS='first.test second.test ...' make -e check
+ You can also run the tests by hand, as explained in the next subsection.
+
+
+About the tests
+---------------
+
+ There are two kinds of tests in the Automake testsuite (both implemented
+ as shell scripts). The scripts with the `.test' suffix are "simple"
+ tests, their outcome completely determined by their exit status. Those
+ with the `.tap' suffix use the TAP protocol. If you want to run a test
+ by hand, you can do so directly if it is a simple test:
+
+ ./nogzip.test
+
+ (it will be verbose by default), while if it is a TAP test you can pass
+ it to your preferred TAP runner, as in e.g.:
+
+ prove --verbose --merge ./add-missing.tap
+
+ The tests can also be run directly in a VPATH build, as with:
+
+ /path/to/srcdir/tests/nogzip.test
+ prove --verbose --merge /path/to/srcdir/tests/add-missing.tap
+
Supported shells
----------------
@@ -82,11 +106,12 @@ Supported shells
`/path/to/another/sh', the user must use:
AM_TESTS_REEXEC=no /path/to/another/sh ./foo.test
+ AM_TESTS_REEXEC=no prove -v -e /path/to/another/sh ./bar.tap
to run a test directly, and:
- make check TEST_LOG_COMPILER=/path/to/sh (GNU make)
- TEST_LOG_COMPILER=/path/to/sh make -e check (non-GNU make)
+ make check LOG_COMPILER=/path/to/sh (GNU make)
+ LOG_COMPILER=/path/to/sh make -e check (non-GNU make)
to run the test(s) through the makefile test driver.
--
1.7.2.3
- [FYI 1/7] {tap-test-protocols} testsuite: run TAP and "plain" tests with the same $(LOG_COMPILER), Stefano Lattarini, 2011/08/07
- [FYI 2/7] {tap-test-protocols} testsuite: run autogenerated tests with $(LOG_COMPILER) too, Stefano Lattarini, 2011/08/07
- [FYI 3/7] {tap-test-protocols} testsuite: TAP tests can have early plan now, Stefano Lattarini, 2011/08/07
- [FYI 4/7] {tap-test-protocols} testsuite: update "user interface" description in tests/README,
Stefano Lattarini <=
- [FYI 5/7] {tap-test-protocols} testsuite: improve and refactor our custom TAP shell library, Stefano Lattarini, 2011/08/07
- [FYI 6/7] {tap-test-protocols} testsuite: TAP tests properly decide when to remove tempdirs, Stefano Lattarini, 2011/08/07
- [FYI 7/7] {tap-test-protocols} test defs: get name of TAP-using tests correctly, Stefano Lattarini, 2011/08/07