Re: [Bug-tar] tar 1.24: OpenBSD testsuite failures

[Paul Eggert]

On 10/27/2010 06:11 PM, John Hein wrote:
> And they all fail at ulimit -n 4
> with 'tar: a: Cannot open: Too many open files'

That's to be expected, as "tar" needs at least 2
file descriptors, one to read and one to write,
plus the 3 standard file descriptors (stdin,
stdout, stderr).

Thanks, Eric, for passing along that suggestion.
I still say the shells in question are buggy, because
a shell should not fail to close a file descriptor
unless close(fd) itself fails, but that's not a bug
we have to fix here.  I installed the following:


>From 8099a072fef24c7039af1ec224187c94e34fdcec Mon Sep 17 00:00:00 2001
From: Paul Eggert <address@hidden>
Date: Wed, 27 Oct 2010 20:25:18 -0700
Subject: [PATCH] tests: port to sh variants that squirrel away file descriptors

OpenBSD /bin/sh, and some other sh variants, squirrel away file
descriptors before closing them.  For example, for "cat 3<&-" they
first dup file descriptor 3 to a fd that is 10 or greater, then
close 3 (because if "cat" had been a builtin command like ":" then
they would have wanted to avoid the fork and restore the fd after
":" finished); and they treat ordinary (forking) commands the same
as builtin commands.  This approach fails after "ulimit -n 10".
Work around this deficiency by closing the file descriptors before
invoking ulimit.  Problem reported by Christian Weisgerber in
<http://lists.gnu.org/archive/html/bug-tar/2010-10/msg00041.html>;
solution suggested by Jilles Tjoelker in
<http://article.gmane.org/gmane.comp.shells.dash/415>.

* tests/extrac11.at (scarce file descriptors): Close file
descriptors before invoking ulimit -n.
---
 tests/extrac11.at |   26 ++++++++++++++++----------
 1 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/tests/extrac11.at b/tests/extrac11.at
index 979b77c..0481c27 100644
--- a/tests/extrac11.at
+++ b/tests/extrac11.at
@@ -46,10 +46,14 @@ for dir in $dirs; do
   done
 done
 
-# Check that "ulimit" itself works.
-( (ulimit -n 100 &&
-   tar -cf archive1.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
-   tar -xf archive1.tar -C dest1 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
+# Check that "ulimit" itself works.  Close file descriptors before
+# invoking ulimit, to work around a bug (or a "feature") in some shells,
+# where they squirrel away dups of file descriptors into FD 10 and up
+# before closing the originals.
+( (exec 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
+   ulimit -n 100 &&
+   tar -cf archive1.tar a &&
+   tar -xf archive1.tar -C dest1 a
   ) &&
   diff -r a dest1/a
 ) >/dev/null 2>&1 ||
@@ -57,18 +61,20 @@ done
 
 # Another test that "ulimit" itself works:
 # tar should fail when completely starved of file descriptors.
-( (ulimit -n 4 &&
-   tar -cf archive2.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
-   tar -xf archive2.tar -C dest2 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
+( (exec 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
+   ulimit -n 4 &&
+   tar -cf archive2.tar a &&
+   tar -xf archive2.tar -C dest2 a
   ) &&
   diff -r a dest2/a
 ) >/dev/null 2>&1 &&
    AT_SKIP_TEST
 
 # Tar should work when there are few, but enough, file descriptors.
-( (ulimit -n 10 &&
-   tar -cf archive3.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
-   tar -xf archive3.tar -C dest3 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
+( (exec 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
+   ulimit -n 10 &&
+   tar -cf archive3.tar a &&
+   tar -xf archive3.tar -C dest3 a
   ) &&
   diff -r a dest3/a >/dev/null 2>&1
 ) || { diff -r a dest3/a; exit 1; }
-- 
1.7.2