bug#7849: new instspc* test failures

[Peter Rosin]

Stefano Lattarini skrev 2012-01-04 21:43:
> Hi Peter, thanks for the super-quick feedback.
> 
> On 01/04/2012 03:17 PM, Peter Rosin wrote:
>> Stefano Lattarini skrev 2012-01-04 13:24:
>>> (Me rummaging in older bug reports ...)
>>>
>>> Reference:
>>>   <http://debbugs.gnu.org/cgi/bugreport.cgi?bug=7849>
>>>
>>> On 01/16/2011 09:46 AM, Ralf Wildenhues wrote:
>>>> There are a number of new failures since the splitup of instspc.test.
>>>> Most seem to stem from increased coverage.  Rather than putting each
>>>> failure in a different PR, I'm listing a number of them here; we can
>>>> still split things out further later if that proves necessary.
>>>>
>>>> On MinGW, instspc-carriageret-* fail because the CR character is not
>>>> treated correctly by the shell and/or other tools like sed.  I don't
>>>> think this is fixable without fixing the shell.
>>>>
>>>> On Cygwin, instspc-dotdotdot-build.test fails because executing a COFF
>>>> binary in a directory named '...' fails (causing the "compiler works"
>>>> test to fail at configure time):
>>>>
>>>> $ mkdir ...
>>>> $ cd ...
>>>> $ cp /usr/bin/ls.exe .
>>>> $ ./ls.exe
>>>> bash: ./ls.exe: No such file or directory
>>>>
>>> Can anyone still reproduce such failures with the latest version of
>>> `instspc.tap' from master?  If yes, can anyone suggest some workarounds
>>> for failures that are due to bugs/limitations in Cygwin and MSYS rather
>>> than to automake bugs?
>>
>> Cygwin runs the test ok these days, but not MSYS/MinGW.
>> MSYS still stumbles on carriageret and the only workaround I could
>> think of is to check if the current test is carriageret and
>> skip instead of fail if $test_string is empty in that case.
>> But that's quite ugly,
>>
> Ah, but no more ugly than many other workarounds we have to avoid spurious
> testsuite failure (if you want to see real horrors, take a look at, e.g.,
> 'uninstall-fail.test', or at 'tap-signal.tap' before commit d47115).

I'd rather not... :-)

>>    eval "test_string=\${instspc__$test_name}" \
>>      && test x"$test_string" != x \
>> -    || fatal_ "invalid test name: '$test_name'"
>> +    || {
>> +      test x"$test_name" != xcarriageret \
>> +        && fatal_ "invalid test name: '$test_name'"
>> +      skip_ "CR probably treated as NL: '$test_name'"
>>
> You need to skip *two* tests here, otherwise the driver will complain that the
> number of tests run doesn't match the TAP plan.  Also, no need to relax the

Ah, right.

> sanity check on the `eval' (that should always work also on  MinGW).  I'd go
> for something like this in the end (untested!):
> 
>    eval "test_string=\${instspc__$test_name}" \
>      || fatal_ "invalid test name: '$test_name'"
> 
>    if test x"$test_string" != x; then
>      if test x"$test_name" != xcarriageret; then
>        fatal_ "invalid test name: '$test_name'"
>      else
>        # MinGW/MSYS version x.y still mishandles carriage returns; see
>        # automake bug#7849.
>        skip_ -r "carriage-return treated as null char" "$test_name in 
> builddir"
>        skip_ -r "carriage-return treated as null char" "$test_name in 
> destddir"
>        continue
>      fi
>    fi

Right, but I fixed a couple of things.

>> and doesn't really bring us much as the test falls over again on
>> dosdrive aka 'a:'.
>>
> Well, it will bring us something once we've fixed also this error :-)
> 
>> On MSYS, "mkdir ./a:" creates
>> the directory "a", w/o the trailing colon. Marvelous.
>>
> Indeed.
> 
>> So, next (ugly as hell) thing to do is to try to create a dir with
>> only a colon if there is any colon in $test_string and skip if that
>> fails.
>>
> Hmm...  what about this simpler sanity check instead?
> 
>   -mkdir "./$test_string" || {
>   +# On MSYS, "mkdir ./a:" creates the directory "a", without the trailing
>   +# colon.  Marvelous.  So, even if mkdir succeeds, we must still check that
>   +# the create directory has the expected name.
>   +mkdir "./$test_string" && test -d "./$test-string" || {
>      skip_ -r "mkdir failed" "$test_name in builddir"
>      skip_ -r "mkdir failed" "$test_name in destdir"
>   ...
> 
> Would this work too?

No, "test -d ..." succeeds. New attempt attached, see below.

>> If you're not busy puking all over the place after reading the
>> patch, you might be able to convince me to write a commit message
>> for it...
>>
> Pretty please? ;-)
> 
>> The test no longer "fatals" on MSYS/MingW with these changes
>> (instspc: exit 0).
>>
> Probably you already know, but I'd rather stress this anyway, just to be sure:
> the TAP-based test scripts do *not* report failure/success through their final
> exit status (any non-zero exit status will be flagged as an hard-error in 
> fact);
> the report of test results will be done with special "directives" emitted on
> the stdout.  So the above result doesn't mean that `instspc.tap' is now 
> passing,
> but only that it is not encountering unexpected errors anymore.
> 
> To correctly execute a TAP-based script and get its results, you must either
> run it through the automake-provided harness:
> 
>   $ make check TESTS=instspc.tap
> 
> or use a proper third-party TAP runner:
> 
>   $ prove --merge ./instspc.tap

Ah, makes perfect sense when you spell it out. Thanks!

Cheers,
Peter

>From 27100f0b94f8e38e8bd30c27277d7ad4e9f4dd1a Mon Sep 17 00:00:00 2001
From: Peter Rosin <address@hidden>
Date: Thu, 5 Jan 2012 01:29:27 +0100
Subject: [PATCH] tests: work around strangeness in MSYS

MSYS mishandles carriage returns and behaves very strange for
directories with colon in them. It seems that colon-directories are
somehow mixed up with drive letters.

Fixes automake bug#7849.

* tests/instspc.tap: Skip instead of erroring out when $test_string
is empty for the carriageret case, as that is expected on MSYS. Also,
for similar reasons, skip instead of erroring out when it is not
possible to cd into the just created directory, and the directory
name contains a colon.
---
 tests/instspc.tap |   28 +++++++++++++++++++++++++++-
 1 files changed, 27 insertions(+), 1 deletions(-)

diff --git a/tests/instspc.tap b/tests/instspc.tap
index 9eb145f..2e9641c 100755
--- a/tests/instspc.tap
+++ b/tests/instspc.tap
@@ -235,9 +235,20 @@ create_input_data
 for test_name in $test_names_list; do
 
   eval "test_string=\${instspc__$test_name}" \
-    && test x"$test_string" != x \
     || fatal_ "invalid test name: '$test_name'"
 
+  if test x"$test_string" = x; then
+    if test x"$test_name" != xcarriageret; then
+      fatal_ "invalid test name: '$test_name'"
+    else
+      # MSYS version 1.0.17 still mishandles carriage returns; see
+      # automake bug#7849.
+      skip_ -r "carriage-return treated as null char" "$test_name in builddir"
+      skip_ -r "carriage-return treated as null char" "$test_name in destdir"
+      continue
+    fi
+  fi
+
   # Skip the next checks if this system doesn't support the required
   # characters in file names.
 
@@ -247,6 +258,21 @@ for test_name in $test_names_list; do
     continue
   }
 
+  case $test_string in
+  *:*)
+    # On MSYS 1.0.17, "mkdir ./a:" creates ./a, and "cd ./a:" takes you
+    # to a strange directory with pwd equal to "a".  But only for
+    # interactive shells.  Or something?  In this script, "cd ./a:" fails
+    # on MSYS.  Marvelous.
+    ( cd "./$test_string" ) || {
+      rmdir "./$test_string" || fatal_ "removing directory"
+      skip_ -r "cd failed" "$test_name in builddir"
+      skip_ -r "cd failed" "$test_name in destdir"
+      continue
+    }
+    ;;
+  esac
+
   # Where are the "weird" characters going to be used, in $(builddir)
   # or in $(DESTDIR)?  They are always going to be used in $(prefix)
   # though; should we maybe separate this into a dedicated check?
-- 
1.7.5.1