ssh setting O_NONBLOCK on stderr can upset CVS

[dhowells]

>Submitter-Id:   net
>Originator:    dhowells@redhat.com 
>Organization:  
net
>Confidential:  no
>Synopsis:      ssh setting O_NONBLOCK on stderr can upset CVS and pcl-cvs
>Severity:      serious
>Priority:      medium
>Category:      cvs
>Class:         sw-bug
>Release:       1.11.1p1
>Environment:   Pentium3 PC, Linux, i686, glibc-2.2.4
System: Linux warthog.cambridge.redhat.com 2.4.7-10 #1 Thu Sep 6 17:27:27 EDT 
2001 i686 unknown
Architecture: i686

>Description:

I've got a problem in which cvs is dangling at the end of a pipe with its
stdin/stderr tied through a pipe to a program that's parsing its output (such
as emacs with its VC minor mode or CVS major mode).

cvs is using ssh to access the CVS server, and this ssh inherits its stderr
file from cvs.

Now ssh sets O_NONBLOCK on its stderr output, and this also affects both stdout
and stderr coming from cvs (since the cvs's fd1 and fd2, and ssh's fd2 all
refer to the same kernel object).

After this has happened, strace shows that cvs starts experiencing writes that
return EAGAIN on its stderr output when the pipe buffer becomes full (which it
just ignores). As far as cvs is concerned, this shouldn't happen (it should,
however, probably check for the error message and retry the write).

ssh's stdout isn't a problem because cvs opens its own pipes for connecting to
ssh's stdin and stdout.

Note that ssh doesn't set the O_NONBLOCK flag on exit either.

As far as strace is concerned, the following two adjacent fcntl64 syscalls are
the only ones that deal with stderr's O_NONBLOCK flag.

ioctl(2, 0x5401, 0xbffff150)            = -1 EINVAL (Invalid argument)
fcntl64(0x2, 0x3, 0, 0x2)               = 1
fcntl64(0x2, 0x4, 0x801, 0x2)           = 0

I'm using openssh version 3.0.1p1 from the RedHat RPM, but it also occurs with
older versions. This bug is registered with the openssh bugzilla system too.

>How-To-Repeat:
        

Import a linux kernel tarball into a cvs server that can be accessed remotely
over SSH, check it out and then start emacs and issue a cvs-examine command
(from pcl-cvs) in the base directory of what you checked out.

As long as ssh/cvs can shovel messages to emacs faster than emacs can read
them, the output pipe connecting the cvs client to emacs will eventually fill
and cvs will start getting EAGAIN errors on writes to stderr (this can be
monitored with strace).

The test system should probably only have a single CPU, otherwise emacs may be
able to empty the pipe as fast as cvs fills it.

>Fix:
        

To work around, wrap ssh using a shell script that interposes a cat in the
stderr pipeline, and point CVS_RSH at this.

To actually correct it, either check for errors on writes to stderr, or create
an extra pipe for dangling ssh from, and use this just to suck ssh errors which
can then be transferred to cvs's stderr.

Note that calling dup(), dup2() and fcntl(F_DUPFD) will not help, since these
just duplicate the attachment to the kernel file object, and not the kernel
file object itself (the O_NONBLOCK/O_NDELAY flag is attached to the file object
not the fd).