Re: [Bug-tar] tar 1.24: OpenBSD testsuite failures

[Paul Eggert]

On 10/26/10 10:25, Christian Weisgerber wrote:

>> cd tests
>> ktrace -df tr make check TESTSUITEFLAGS=36
> 
> You mean "ktrace -i" (inherit; pass the trace flags to all future
> children of the designated processes), I think.

Thanks.  In looking at the ktrace output it appears to be a bug in
OpenBSD, not in tar itself.  The relevant shell source code (in
testsuite) is:

# Tar should work when there are few, but enough, file descriptors.
((ulimit -n 10 &&
  tar -cf archive3.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
  tar -xf archive3.tar -C dest3 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
 ) &&
 diff -r a dest3/a >/dev/null 2>&1
) || { diff -r a dest3/a; exit 1; }
)"

The ktrace/kdump output in the neighborhood looks like this:

 10306 sh       CALL  fcntl(0x3,0,0xa)
 10306 sh       RET   fcntl -1 errno 9 Bad file descriptor
 10306 sh       CALL  close(0x3)
 10306 sh       RET   close -1 errno 9 Bad file descriptor
 10306 sh       CALL  fcntl(0x4,0,0xa)
 10306 sh       RET   fcntl -1 errno 9 Bad file descriptor
 10306 sh       CALL  close(0x4)
 10306 sh       RET   close -1 errno 9 Bad file descriptor
 10306 sh       CALL  fcntl(0x5,0,0xa)
 10306 sh       RET   fcntl -1 errno 22 Invalid argument
 10306 sh       CALL  write(0x2,0x83fc6c08,0x80)
 10306 sh       GIO   fd 2 wrote 128 bytes
       "./testsuite: /usr/ports/pobj/gtar-1.24/tar-1.24/tests/testsuite.dir/at\
        -groups/36/test-source[209]: too many files open in shell
       "

If I've decoded that correctly, the shell is attempting to implement
3<&- by duping file descriptor 3 to something above file descriptor
10.  (It's weird to dup before closing, but never mind.)  The shell can't
do this, because of the ulimit -n 10 in the script.  I suspect that
the OpenBSD shell is fooling around with file descriptors, and
caching low numbered file descriptors by duping them above fd 10, for
whatever reason.  The problem is compounded by the dup call
(fcntl, here) returning "Invalid argument" for some reason.  And it's
compounded further by the shell turning the "Invalid argument" errno
into "too many files open in shell".

I expect that the best way to work around this is to disable the test
if the shell has this problem.  Is there some simple way to test this?

For example, what does the following little shell script do on
your host?

#! /bin/sh
if (ulimit -n 10 && true 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-) </dev/null 
>/dev/null 2>&1
then echo OK
else echo buggy shell
fi

If this exits with nonzero status on your host, or does something
odd like that, then we should be able to modify the test so that
it's skipped when the shell has the problem in question.  Or perhaps you
can come up with a simple variant of the above test that will do the
trick (maybe replace "true" with "cat /dev/null"?).  Thanks for any
help that you can provide in this area, as I don't use OpenBSD.

In reviewing the test, I see another possible portability problem, and
I pushed the following patch.  I doubt whether this will fix your
issue, though.

>From 227698e52006a99a3935a4033d63b0bb1c889d30 Mon Sep 17 00:00:00 2001
From: Paul Eggert <address@hidden>
Date: Tue, 26 Oct 2010 11:33:23 -0700
Subject: [PATCH] tar: don't assume stdin is open when testing fd limits

* tests/extrac11.at: Redirect stdin from /dev/null, in case
the parent 'make' is running with stdin closed.
---
 tests/extrac11.at |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/tests/extrac11.at b/tests/extrac11.at
index 9456695..f7ba587 100644
--- a/tests/extrac11.at
+++ b/tests/extrac11.at
@@ -24,6 +24,7 @@ AT_SETUP([scarce file descriptors])
 AT_KEYWORDS([extract extrac11])
 
 AT_TAR_CHECK([
+exec </dev/null
 dirs='a
       a/b
       a/b/c
-- 
1.7.2