Spurious failures with -x autotest option

[Noah Misch]

Consider this comment in lib/autotest/general.m4 concerning behavior when the
user passes -x to a test suite:

# Any line of stderr starting with leading blanks and a `+' are filtered
# out, since most shells when tracing include subshell traces in stderr.
# This may cause spurious failures when the test suite is run with `-x'.

One specific case I have found to trigger a spurious failure is an AT_CHECK of
the standard error of a multi-line command.  Only the first line of the shell
trace of the command gets a leading ``+'', so the grep that filters out shell
trace lines misses the remaining lines.  Consider:

% bash
$ set -x
$ echo foo'
> 'bar
+ echo 'foo
bar'
foo
bar
$ set +x
$ ( set -x; echo foo'
> 'bar ) 2>&1 | grep -v ^+
bar'
foo
bar

With -x, tests 4, 54, 55, 56, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89,
91, and 95 to fail on my Fedora Core 1 box, where no test fails without -x.

I tried to fix this by counting the lines in the supplied command and stripping
that many from stderr to account for the shell trace.  This (bad) patch against
CVS HEAD lib/autotest/general.m4 illustrates that approach:

--- general.m4-orig     2004-06-25 03:44:45.000000000 -0400
+++ general.m4-wc       2004-06-25 22:52:19.000000000 -0400
@@ -1193,11 +1190,22 @@
 m4_define([_AT_CHECK],
 [$at_traceoff
 echo "AT_LINE: AS_ESCAPE([$1])"
 echo AT_LINE >$at_check_line_file
-( $at_traceon; $1 ) >$at_stdout 2>$at_stder1
-at_status=$?
-grep '^ *+' $at_stder1 >&2
-grep -v '^ *+' $at_stder1 >$at_stderr
+if test -n "$at_traceon"; then
+    cat <<'_EOF__AT_CHECK' | sed '/\\$/ d' | wc -l >$at_wc_file
+$1
+_EOF__AT_CHECK
+    at_cmd_wc=`cat $at_wc_file`
+    at_cmd_extra=`expr $at_cmd_wc - 1`
+    ( $at_traceon; $1 ) >$at_stdout 2>$at_stder1
+    at_status=$?
+    sed -n "/^ *+/,+${at_cmd_extra}p" $at_stder1 >&2
+    sed    "/^ *+/,+${at_cmd_extra}d" $at_stder1 >$at_stderr
+else
+    ( $1 ) >$at_stdout 2>$at_stderr
+    at_status=$?
+fi
 at_failed=false
 dnl Check stderr.
 m4_case([$4],

This strategy has a few problems, but most severely, it does not handle a
situation like this:

# earlier in the program
a=foo'
'bar

# command to test
/bin/echo "$a"

The quoted here-document will contain a single line, but the shell trace line
will include the parameter expansion and therefore span two lines.  One could
fix this case by removing the quotes on the here-document delimiter, but that
breaks handling of another class of commands:

/bin/echo `cat /etc/services`

When the shell expands that string in the here-document, it will perform
parameter expansion, command substitution, and quote removal, yielding as many
lines as /etc/services contains.  When the shell executes that line, however, it
will also perform field splitting on the result of the command substitution,
yielding only a single line.  The trace output will have only that single line.

I tried to think of a robust work-around, but I found none.  Have I missed any
good solutions?

If not, would it make sense to disable shell tracing of commands for which we
examine stderr?  This patch shows how I would approach it; if you like the idea,
I'll write a ChangeLog entry and modify the patch to document the new behavior
in the manual and delete comments it obsoletes.

I get zero Autoconf test suite failures with this patch and -x.

Index: general.m4
===================================================================
RCS file: /cvsroot/autoconf/autoconf/lib/autotest/general.m4,v
retrieving revision 1.156
diff -u -r1.156 general.m4
--- general.m4  30 Jan 2004 14:29:18 -0000      1.156
+++ general.m4  26 Jun 2004 04:19:11 -0000
@@ -1193,11 +1191,14 @@
 m4_define([_AT_CHECK],
 [$at_traceoff
 echo "AT_LINE: AS_ESCAPE([$1])"
 echo AT_LINE >$at_check_line_file
-( $at_traceon; $1 ) >$at_stdout 2>$at_stder1
+m4_if([$4], [ignore],
+[( $at_traceon; $1 ) >$at_stdout 2>$at_stderr],
+[if test -n "$at_traceon"; then
+    echo 'This test examines its stderr; not enabling shell tracing.'
+fi
+( $1 ) >$at_stdout 2>$at_stderr])
 at_status=$?
-grep '^ *+' $at_stder1 >&2
-grep -v '^ *+' $at_stder1 >$at_stderr
 at_failed=false
 dnl Check stderr.
 m4_case([$4],