Re: [Bug-tar] tar 1.24: OpenBSD testsuite failures

[Christian Weisgerber]

(Excuse the extensive quoting.)

Paul Eggert:

> # Tar should work when there are few, but enough, file descriptors.
> ((ulimit -n 10 &&
>   tar -cf archive3.tar a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&- &&
>   tar -xf archive3.tar -C dest3 a 3<&- 4<&- 5<&- 6<&- 7<&- 8<&- 9<&-
>  ) &&
>  diff -r a dest3/a >/dev/null 2>&1
> ) || { diff -r a dest3/a; exit 1; }
> )"
> 
> The ktrace/kdump output in the neighborhood looks like this:
> 
>  10306 sh       CALL  fcntl(0x3,0,0xa)
>  10306 sh       RET   fcntl -1 errno 9 Bad file descriptor
>  10306 sh       CALL  close(0x3)
>  10306 sh       RET   close -1 errno 9 Bad file descriptor
>  10306 sh       CALL  fcntl(0x4,0,0xa)
>  10306 sh       RET   fcntl -1 errno 9 Bad file descriptor
>  10306 sh       CALL  close(0x4)
>  10306 sh       RET   close -1 errno 9 Bad file descriptor
>  10306 sh       CALL  fcntl(0x5,0,0xa)
>  10306 sh       RET   fcntl -1 errno 22 Invalid argument
>  10306 sh       CALL  write(0x2,0x83fc6c08,0x80)
>  10306 sh       GIO   fd 2 wrote 128 bytes
>        "./testsuite: 
> /usr/ports/pobj/gtar-1.24/tar-1.24/tests/testsuite.dir/at\
>       -groups/36/test-source[209]: too many files open in shell
>        "
> 
> If I've decoded that correctly, the shell is attempting to implement
> 3<&- by duping file descriptor 3 to something above file descriptor
> 10.  (It's weird to dup before closing, but never mind.)  The shell can't
> do this, because of the ulimit -n 10 in the script.  I suspect that
> the OpenBSD shell is fooling around with file descriptors, and
> caching low numbered file descriptors by duping them above fd 10, for
> whatever reason.  The problem is compounded by the dup call
> (fcntl, here) returning "Invalid argument" for some reason.  And it's
> compounded further by the shell turning the "Invalid argument" errno
> into "too many files open in shell".

I've traced some simple calls like ": 3<&-" etc and your analysis
is basically correct.  The shell tries to do

 22157 sh       CALL  fcntl(0x2,0,0xa)      # fcntl(fd, F_DUPFD, 10)
 22157 sh       RET   fcntl 10/0xa
 22157 sh       CALL  fcntl(0xa,0x2,0x1)    # fcntl(nfd, F_SETFD, FD_CLOEXEC)
 22157 sh       RET   fcntl 0
 22157 sh       CALL  close(0x2)
 22157 sh       RET   close 0
 22157 sh       CALL  dup2(0xa,0x2)
 22157 sh       RET   dup2 2
 22157 sh       CALL  close(0xa)
 22157 sh       RET   close 0

when closing.  If the descriptor is not open, EBADF is returned,
regardless of the limit.  If the descriptor is actually in use, but
the dup fails because of the limit, EINVAL is returned and we get
the "too many files open in shell" error message.

> I expect that the best way to work around this is to disable the test
> if the shell has this problem.  Is there some simple way to test this?

This produces "too many files open in shell" right away:

ulimit -n 10; : <&-

-- 
Christian "naddy" Weisgerber                          address@hidden