[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PATCH] {master} Extend and improve tests on DejaGnu support.
From: |
Stefano Lattarini |
Subject: |
[PATCH] {master} Extend and improve tests on DejaGnu support. |
Date: |
Sun, 14 Nov 2010 14:29:33 +0100 |
User-agent: |
KMail/1.13.3 (Linux/2.6.30-2-686; KDE/4.4.4; i686; ; ) |
Another testsuite patch, based off of maint, but to be merged
in master only.
I will wait the customary 72 hours before pushing.
Regards,
Stefano
-*-*-*-
Extend and improve tests on DejaGnu support.
* tests/dejagnu.test: Do not create useless dummy test script.
Add trailing `:' command. In heading comments, add reference
to ...
* tests/check12.test: ... this new "semantic" test, covering
concurrent use of dejagnu tests, simple tests and `check-local'
target.
* tests/dejagnu2.test: Make test more reliable, by avoid weak
grepping of make output. Prefer `cat' over `echo' to append
to configure.in. Quote literal dots in grep regexps. Prefer
`grep -c ...' over `grep ... | wc -l'. Make grepping of
automake stderr slighty stricter. Add trailing `:' command.
* tests/dejagnu3.test: Prefer `cat' over `echo' to append to
configure.in. Check stderr of expected-to-fail "make" call.
Remove extra blank lines from Makefile.am.
* tests/dejagnu4.test: Prefer `cat' over `echo' to append to
configure.in. Prefer `mv -f' over plain `mv' when the target
file already exists. Avoid extra mkdir calls by creating more
directories at once. Better use of blank lines. Check that
the `*.log' and `*.sum' files are created by runtest also when
"make check" fails.
* tests/dejagnu7.test: Prefer `cat' over `echo' to append to
configure.in. Better use of blank lines. Add a trailing `:'
command.
* tests/dejagnu6.test: Likewise, and give the dejagnu test a
more descriptive name.
* tests/dejagnu5.test: Likewise. Also, simply define package
name to `$me' rather than using a non-obvious sed script to
extract it from `AC_INIT', and write the Makefile.am with only
one command.
* tests/Makefile.am (TESTS): Updated.
---
ChangeLog | 34 +++++++++
tests/Makefile.am | 1 +
tests/Makefile.in | 2 +
tests/check12.test | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dejagnu.test | 7 +-
tests/dejagnu2.test | 21 +++---
tests/dejagnu3.test | 17 +++--
tests/dejagnu4.test | 19 +++--
tests/dejagnu5.test | 19 +++---
tests/dejagnu6.test | 16 ++--
tests/dejagnu7.test | 10 ++--
11 files changed, 298 insertions(+), 49 deletions(-)
create mode 100755 tests/check12.test
From 39ab915725c5eb5fe055dd3e09116982b1abcce1 Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Sat, 16 Oct 2010 17:03:03 +0200
Subject: [PATCH] Extend and improve tests on DejaGnu support.
* tests/dejagnu.test: Do not create useless dummy test script.
Add trailing `:' command. In heading comments, add reference
to ...
* tests/check12.test: ... this new "semantic" test, covering
concurrent use of dejagnu tests, simple tests and `check-local'
target.
* tests/dejagnu2.test: Make test more reliable, by avoid weak
grepping of make output. Prefer `cat' over `echo' to append
to configure.in. Quote literal dots in grep regexps. Prefer
`grep -c ...' over `grep ... | wc -l'. Make grepping of
automake stderr slighty stricter. Add trailing `:' command.
* tests/dejagnu3.test: Prefer `cat' over `echo' to append to
configure.in. Check stderr of expected-to-fail "make" call.
Remove extra blank lines from Makefile.am.
* tests/dejagnu4.test: Prefer `cat' over `echo' to append to
configure.in. Prefer `mv -f' over plain `mv' when the target
file already exists. Avoid extra mkdir calls by creating more
directories at once. Better use of blank lines. Check that
the `*.log' and `*.sum' files are created by runtest also when
"make check" fails.
* tests/dejagnu7.test: Prefer `cat' over `echo' to append to
configure.in. Better use of blank lines. Add a trailing `:'
command.
* tests/dejagnu6.test: Likewise, and give the dejagnu test a
more descriptive name.
* tests/dejagnu5.test: Likewise. Also, simply define package
name to `$me' rather than using a non-obvious sed script to
extract it from `AC_INIT', and write the Makefile.am with only
one command.
* tests/Makefile.am (TESTS): Updated.
---
ChangeLog | 34 +++++++++
tests/Makefile.am | 1 +
tests/Makefile.in | 2 +
tests/check12.test | 201 +++++++++++++++++++++++++++++++++++++++++++++++++++
tests/dejagnu.test | 7 +-
tests/dejagnu2.test | 21 +++---
tests/dejagnu3.test | 17 +++--
tests/dejagnu4.test | 19 +++--
tests/dejagnu5.test | 19 +++---
tests/dejagnu6.test | 16 ++--
tests/dejagnu7.test | 10 ++--
11 files changed, 298 insertions(+), 49 deletions(-)
create mode 100755 tests/check12.test
diff --git a/ChangeLog b/ChangeLog
index a356958..4fd15f1 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,37 @@
+2010-11-14 Stefano Lattarini <address@hidden>
+
+ Extend and improve tests on DejaGnu support.
+ * tests/dejagnu.test: Do not create useless dummy test script.
+ Add trailing `:' command. In heading comments, add reference
+ to ...
+ * tests/check12.test: ... this new "semantic" test, covering
+ concurrent use of dejagnu tests, simple tests and `check-local'
+ target.
+ * tests/dejagnu2.test: Make test more reliable, by avoid weak
+ grepping of make output. Prefer `cat' over `echo' to append
+ to configure.in. Quote literal dots in grep regexps. Prefer
+ `grep -c ...' over `grep ... | wc -l'. Make grepping of
+ automake stderr slighty stricter. Add trailing `:' command.
+ * tests/dejagnu3.test: Prefer `cat' over `echo' to append to
+ configure.in. Check stderr of expected-to-fail "make" call.
+ Remove extra blank lines from Makefile.am.
+ * tests/dejagnu4.test: Prefer `cat' over `echo' to append to
+ configure.in. Prefer `mv -f' over plain `mv' when the target
+ file already exists. Avoid extra mkdir calls by creating more
+ directories at once. Better use of blank lines. Check that
+ the `*.log' and `*.sum' files are created by runtest also when
+ "make check" fails.
+ * tests/dejagnu7.test: Prefer `cat' over `echo' to append to
+ configure.in. Better use of blank lines. Add a trailing `:'
+ command.
+ * tests/dejagnu6.test: Likewise, and give the dejagnu test a
+ more descriptive name.
+ * tests/dejagnu5.test: Likewise. Also, simply define package
+ name to `$me' rather than using a non-obvious sed script to
+ extract it from `AC_INIT', and write the Makefile.am with only
+ one command.
+ * tests/Makefile.am (TESTS): Updated.
+
2010-11-07 Stefano Lattarini <address@hidden>
Fix a bug in variable concatanation with `+='.
diff --git a/tests/Makefile.am b/tests/Makefile.am
index da81c49..9d8cd81 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -133,6 +133,7 @@ check8.test \
check9.test \
check10.test \
check11.test \
+check12.test \
checkall.test \
clean.test \
clean2.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index eb461a9..8e14e1e 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -283,6 +283,7 @@ parallel_tests = \
check-p.test \
check10-p.test \
check11-p.test \
+check12-p.test \
check2-p.test \
check3-p.test \
check4-p.test \
@@ -400,6 +401,7 @@ check8.test \
check9.test \
check10.test \
check11.test \
+check12.test \
checkall.test \
clean.test \
clean2.test \
diff --git a/tests/check12.test b/tests/check12.test
new file mode 100755
index 0000000..8be6123
--- /dev/null
+++ b/tests/check12.test
@@ -0,0 +1,201 @@
+#! /bin/sh
+# Copyright (C) 2010 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+# Test to make sure dejagnu tests, automake-style tests, and check-local
+# target can coexist.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS =
+EXTRA_DIST =
+CLEANFILES =
+END
+
+## Simple tests.
+
+cat > a.test << 'END'
+#!/bin/sh
+exit ${A_EXIT_STATUS-0}
+END
+cat > b.test << 'END'
+#!/bin/sh
+exit ${B_EXIT_STATUS-0}
+END
+chmod +x a.test b.test
+
+cat >> Makefile.am << 'END'
+TESTS = a.test b.test
+EXTRA_DIST += $(TESTS)
+END
+
+A_EXIT_STATUS=0; export A_EXIT_STATUS
+B_EXIT_STATUS=0; export B_EXIT_STATUS
+
+## DejaGnu tests.
+
+cat >> Makefile.am << 'END'
+AUTOMAKE_OPTIONS += dejagnu
+DEJATOOL = hammer spanner
+AM_RUNTESTFLAGS = HAMMER=$(srcdir)/hammer SPANNER=$(srcdir)/spanner
+EXTRA_DIST += hammer.test/hammer.exp
+EXTRA_DIST += spanner.test/spanner.exp
+END
+
+cat > hammer << 'END'
+#! /bin/sh
+echo "Everything looks like a ${NAIL-nail} to me!"
+END
+
+NAIL=nail; export NAIL
+
+cat > spanner << 'END'
+#! /bin/sh
+echo "I'm a right spanner!"
+END
+chmod +x hammer spanner
+
+mkdir hammer.test spanner.test
+
+cat > hammer.test/hammer.exp << 'END'
+set test test_hammer
+spawn $HAMMER
+expect {
+ "Everything looks like a nail to me!" { pass "$test" }
+ default { fail "$test" }
+}
+END
+
+cat > spanner.test/spanner.exp << 'END'
+set test test_spanner
+spawn $SPANNER
+expect {
+ "I'm a right spanner!" { pass "$test" }
+ default { fail "$test" }
+}
+END
+
+## User-defined extra tests.
+
+cat >> Makefile.am <<'END'
+check-local:
+ case $$CHECKLOCAL_EXIT_STATUS in \
+ 0) echo 'check-local succeded :-)';; \
+ *) echo 'check-local failed :-(';; \
+ esac >local.log
+ exit $$CHECKLOCAL_EXIT_STATUS
+CLEANFILES += local.log
+END
+CHECKLOCAL_EXIT_STATUS=0; export CHECKLOCAL_EXIT_STATUS
+
+## Go with the testcase execution.
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE --add-missing
+
+for vpath in : false; do
+
+ if $vpath; then
+ srcdir=..
+ mkdir build_
+ cd build_
+ else
+ srcdir=.
+ fi
+
+ $srcdir/configure
+
+ $MAKE check
+ test -f hammer.log
+ test -f hammer.sum
+ test -f spanner.log
+ test -f spanner.sum
+ # This checks will be run only by the autogenerated `check12-p.test'.
+ if test x"$parallel_tests" = x"yes"; then
+ cat test-suite.log
+ grep '^PASS: a\.test (exit: 0)' a.log
+ grep '^PASS: b\.test (exit: 0)' b.log
+ else
+ :
+ fi
+ grep 'check-local succeded :-)' local.log
+
+ cp -f config.status config-status.sav
+
+ $MAKE distclean
+ test ! -r hammer.log
+ test ! -r hammer.sum
+ test ! -r spanner.log
+ test ! -r spanner.sum
+ test ! -r test-suite.log
+ test ! -r a.log
+ test ! -r b.log
+ test ! -r local.log
+
+ mv -f config-status.sav config.status
+ ./config.status
+
+ NAIL=screw $MAKE check && Exit 1
+ test -f hammer.log
+ test -f hammer.sum
+ test -f spanner.log
+ test -f spanner.sum
+ grep 'FAIL: test_hammer' hammer.sum
+ grep 'FAIL:' spanner.sum && Exit 1
+
+ B_EXIT_STATUS=1 $MAKE check && Exit 1
+ # This checks will be run only by the autogenerated `check12-p.test'.
+ if test x"$parallel_tests" = x"yes"; then
+ cat test-suite.log
+ grep '^PASS: a\.test (exit: 0)' a.log
+ grep '^FAIL: b\.test (exit: 1)' b.log
+ grep '^FAIL: b\.test (exit: 1)' test-suite.log
+ else :; fi
+
+ CHECKLOCAL_EXIT_STATUS=1 $MAKE check && Exit 1
+ grep 'check-local failed :-(' local.log
+
+ # Do not trust the exit status of `make -k'.
+ NAIL=screw B_EXIT_STATUS=23 CHECKLOCAL_EXIT_STATUS=1 $MAKE -k check || :
+ test -f hammer.log
+ test -f hammer.sum
+ test -f spanner.log
+ test -f spanner.sum
+ grep 'FAIL: test_hammer' hammer.sum
+ grep 'FAIL:' spanner.sum && Exit 1
+ # This checks will be run only by the autogenerated `check12-p.test'.
+ if test x"$parallel_tests" = x"yes"; then
+ cat test-suite.log
+ grep '^PASS: a\.test (exit: 0)' a.log
+ grep '^FAIL: b\.test (exit: 23)' b.log
+ grep '^FAIL: b\.test (exit: 23)' test-suite.log
+ else :; fi
+ grep 'check-local failed :-(' local.log
+
+ $MAKE distcheck
+ $MAKE distclean
+ cd $srcdir
+
+done
+
+:
diff --git a/tests/dejagnu.test b/tests/dejagnu.test
index a273770..b3e58e7 100755
--- a/tests/dejagnu.test
+++ b/tests/dejagnu.test
@@ -14,7 +14,8 @@
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
-# Test to make sure dejagnu tests and automake-style tests can coexist.
+# Superficial test to check that dejagnu tests and automake-style
+# tests can coexist. See also related deeper test `check12.test'.
. ./defs || Exit 1
@@ -25,10 +26,10 @@ AUTOMAKE_OPTIONS = dejagnu
TESTS = frob.test
END
-: > frob.test
-
$ACLOCAL
$AUTOMAKE
grep '^check-TESTS' Makefile.in
grep '^check-DEJAGNU' Makefile.in
+
+:
diff --git a/tests/dejagnu2.test b/tests/dejagnu2.test
index 531f744..121f6d8 100755
--- a/tests/dejagnu2.test
+++ b/tests/dejagnu2.test
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2001, 2002, 2003 Free Software Foundation, Inc.
+# Copyright (C) 2001, 2002, 2003, 2010 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -20,26 +20,29 @@
set -e
-echo 'AC_OUTPUT' >> configure.in
+cat >> configure.in <<'END'
+AC_OUTPUT
+END
cat > Makefile.am << 'END'
AUTOMAKE_OPTIONS = dejagnu
site.exp:
- @echo foo
+ echo :GREP:ME: > $@
END
$ACLOCAL
$AUTOCONF
$AUTOMAKE -Wno-override
-grep site.exp Makefile.in
-test `grep '^site\.exp:' Makefile.in | wc -l` -eq 1
+grep 'site\.exp' Makefile.in
+test `grep -c '^site\.exp:' Makefile.in` -eq 1
./configure
-$MAKE site.exp >stdout
-cat stdout
-grep foo stdout
+$MAKE site.exp
+grep ':GREP:ME:' site.exp
AUTOMAKE_fails
-grep 'Makefile.am:3:.*site.exp' stderr
+grep '^Makefile\.am:3:.*site\.exp' stderr
+
+:
diff --git a/tests/dejagnu3.test b/tests/dejagnu3.test
index d812ac4..2c5d375 100755
--- a/tests/dejagnu3.test
+++ b/tests/dejagnu3.test
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2003 Free Software Foundation, Inc.
+# Copyright (C) 2003, 2010 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -28,15 +28,14 @@ END
chmod +x hammer
-echo AC_OUTPUT >> configure.in
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
cat > Makefile.am << 'END'
AUTOMAKE_OPTIONS = dejagnu
-
DEJATOOL = hammer
-
AM_RUNTESTFLAGS = HAMMER=$(srcdir)/hammer
-
EXTRA_DIST = hammer hammer.test/hammer.exp
END
@@ -65,5 +64,9 @@ $MAKE distcheck
# Ensure that the envvar RUNTESTFLAGS is used.
# Report from Mark Mitchell.
-RUNTESTFLAGS=--unknown-option $MAKE check && Exit 1
-Exit 0
+RUNTESTFLAGS=--unknown-runtest-option $MAKE check >output 2>&1 \
+ && { cat output; Exit 1; }
+cat output
+$FGREP 'unknown-runtest-option' output
+
+:
diff --git a/tests/dejagnu4.test b/tests/dejagnu4.test
index 5fdda10..fa64a31 100755
--- a/tests/dejagnu4.test
+++ b/tests/dejagnu4.test
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2003, 2006 Free Software Foundation, Inc.
+# Copyright (C) 2003, 2006, 2010 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -26,17 +26,17 @@ cat > hammer << 'END'
#! /bin/sh
echo "Everything looks like a nail to me!"
END
-
chmod +x hammer
cat > spanner << 'END'
#! /bin/sh
echo "I'm a right spanner!"
END
-
chmod +x spanner
-echo AC_OUTPUT >> configure.in
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
cat > Makefile.am << 'END'
AUTOMAKE_OPTIONS = dejagnu
@@ -49,8 +49,7 @@ EXTRA_DIST = hammer hammer.test/hammer.exp
EXTRA_DIST += spanner spanner.test/spanner.exp
END
-mkdir hammer.test
-mkdir spanner.test
+mkdir hammer.test spanner.test
cat > hammer.test/hammer.exp << 'END'
set test test
@@ -86,8 +85,14 @@ $MAKE distcheck
# Test for PR 488.
sed 's/E\(verything\)/Not e\1/' hammer > thammer
-mv thammer hammer
+mv -f thammer hammer
chmod +x hammer
+rm -f hammer.log hammer.sum spanner.log spanner.sum
$MAKE check && Exit 1
+test -f hammer.log
+test -f hammer.sum
+test -f spanner.log
+test -f spanner.sum
+
:
diff --git a/tests/dejagnu5.test b/tests/dejagnu5.test
index 0095958..0237a08 100755
--- a/tests/dejagnu5.test
+++ b/tests/dejagnu5.test
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2003 Free Software Foundation, Inc.
+# Copyright (C) 2003, 2010 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -22,30 +22,27 @@ required=runtest
set -e
-# Get the package name from configure.in.
-package=`$FGREP AC_INIT configure.in | sed -e 's:^[^[]*\[\([^]]*\)\].*$:\1:'`
+package=$me
cat > $package << 'END'
#! /bin/sh
echo "Ah, we have been expecting you, Mr. Blond."
END
-
chmod +x $package
-echo AC_OUTPUT >> configure.in
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
cat > Makefile.am << END
AUTOMAKE_OPTIONS = dejagnu
-
EXTRA_DIST = $package $package.test/$package.exp
+AM_RUNTESTFLAGS = PACKAGE=\$(srcdir)/$package
END
-echo "AM_RUNTESTFLAGS = PACKAGE=\$(srcdir)/$package" >> Makefile.am
-
mkdir $package.test
-
cat > $package.test/$package.exp << 'END'
-set test "test"
+set test "a_dejagnu_test"
spawn $PACKAGE
expect {
"Ah, we have been expecting you, Mr. Blond." { pass "$test" }
@@ -64,3 +61,5 @@ test -f $package.log
test -f $package.sum
$MAKE distcheck
+
+:
diff --git a/tests/dejagnu6.test b/tests/dejagnu6.test
index 25a1f1f..662ca28 100755
--- a/tests/dejagnu6.test
+++ b/tests/dejagnu6.test
@@ -1,5 +1,5 @@
#! /bin/sh
-# Copyright (C) 2003 Free Software Foundation, Inc.
+# Copyright (C) 2003, 2010 Free Software Foundation, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -25,23 +25,21 @@ cat > faildeja << 'END'
#! /bin/sh
echo whatever
END
-
chmod +x faildeja
-echo AC_OUTPUT >> configure.in
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
cat > Makefile.am << 'END'
AUTOMAKE_OPTIONS = dejagnu
-
DEJATOOL = faildeja
-
AM_RUNTESTFLAGS = FAILDEJA=$(srcdir)/faildeja
END
mkdir faildeja.test
-
cat > faildeja.test/faildeja.exp << 'END'
-set test test
+set test failing_deja_test
spawn $FAILDEJA
expect {
default { fail "$test" }
@@ -57,4 +55,6 @@ $AUTOMAKE --add-missing
$MAKE check && Exit 1
test -f faildeja.log
test -f faildeja.sum
-$FGREP 'FAIL: test' faildeja.sum
+$FGREP 'FAIL: failing_deja_test' faildeja.sum
+
+:
diff --git a/tests/dejagnu7.test b/tests/dejagnu7.test
index 552c2ca..0be4d1f 100755
--- a/tests/dejagnu7.test
+++ b/tests/dejagnu7.test
@@ -29,21 +29,19 @@ cat > failtcl << 'END'
#! /bin/sh
echo whatever
END
-
chmod +x failtcl
-echo AC_OUTPUT >> configure.in
+cat >> configure.in << 'END'
+AC_OUTPUT
+END
cat > Makefile.am << 'END'
AUTOMAKE_OPTIONS = dejagnu
-
DEJATOOL = failtcl
-
AM_RUNTESTFLAGS = --status FAILTCL=$(srcdir)/failtcl
END
mkdir failtcl.test
-
cat > failtcl.test/failtcl.exp << 'END'
set test test
spawn $FAILTCL
@@ -62,3 +60,5 @@ $MAKE check && Exit 1
test -f failtcl.log
test -f failtcl.sum
$FGREP 'missing close-brace' failtcl.sum
+
+:
--
1.7.1
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [PATCH] {master} Extend and improve tests on DejaGnu support.,
Stefano Lattarini <=