Re: handling of closed file descriptors

[Bruno Haible]

I wrote:
> I would like to use this modules in GNU gettext and GNU libiconv.

Well, I thought that most programs do need one form of the mitigation
against the closed file descriptors. I'm surprised to see not the case
for any of the programs from GNU gettext and GNU libiconv.

This deserves some documentation. I'm adding this new section.


2019-01-06  Bruno Haible  <address@hidden>

        doc: Document the xstdopen and *-safer modules.
        * doc/xstdopen.texi: New file.
        * doc/gnulib.texi (Particular Modules): Include it.

diff --git a/doc/gnulib.texi b/doc/gnulib.texi
index a79c7c9..ef1cda3 100644
--- a/doc/gnulib.texi
+++ b/doc/gnulib.texi
@@ -6365,6 +6365,7 @@ to POSIX that it can be treated like any other Unix-like 
platform.
 * Compile-time Assertions::
 * Integer Properties::
 * extern inline::
+* Closed standard fds::
 * String Functions in C Locale::
 * Quoting::
 * error and progname::
@@ -6394,6 +6395,8 @@ to POSIX that it can be treated like any other Unix-like 
platform.
 
 @include extern-inline.texi
 
address@hidden xstdopen.texi
+
 @include c-locale.texi
 
 @include quote.texi
diff --git a/doc/xstdopen.texi b/doc/xstdopen.texi
new file mode 100644
index 0000000..f68f85b
--- /dev/null
+++ b/doc/xstdopen.texi
@@ -0,0 +1,226 @@
address@hidden GNU xstdopen and *-safer modules documentation
+
address@hidden Copyright (C) 2019 Free Software Foundation, Inc.
+
address@hidden Permission is granted to copy, distribute and/or modify this 
document
address@hidden under the terms of the GNU Free Documentation License, Version 
1.3
address@hidden or any later version published by the Free Software Foundation;
address@hidden with no Invariant Sections, no Front-Cover Texts, and no 
Back-Cover
address@hidden Texts.  A copy of the license is included in the ``GNU Free
address@hidden Documentation License'' file as part of this distribution.
+
address@hidden Written by Bruno Haible, based on ideas from Paul Eggert.
+
address@hidden Closed standard fds
address@hidden Handling closed standard file descriptors
+
address@hidden xstdopen
address@hidden stdopen
address@hidden dirent-safer
address@hidden fcntl-safer
address@hidden fopen-safer
address@hidden freopen-safer
address@hidden openat-safer
address@hidden pipe2-safer
address@hidden popen-safer
address@hidden stdlib-safer
address@hidden tmpfile-safer
address@hidden unistd-safer
+
+Usually, when a program gets invoked, its file descriptors
+0 (for standard input), 1 (for standard output), and 2 (for standard error)
+are open.  But there are situations when some of these file descriptors are
+closed.  These situations can arise when
address@hidden @bullet
address@hidden
+The invoking process invokes @code{close()} on the file descriptor before
address@hidden, or
address@hidden
+The invoking process invokes @code{posix_spawn_file_actions_addclose()} for
+the file descriptor before @code{posix_spawn} or @code{posix_spawnp}, or
address@hidden
+The invoking process is a Bourne shell, and the shell script uses the
+POSIX syntax for closing the file descriptor:
address@hidden<&-} for closing standard input,
address@hidden>&-} for closing standard output, or
address@hidden>&-} for closing standard error.
address@hidden itemize
+
+When a closed file descriptor is accessed through a system call, such as
address@hidden()}, @code{fstat()}, @code{read()}, or @code{write()}, the
+system calls fails with error @code{EBADF} ("Bad file descriptor").
+
+When a new file descriptor is allocated, the operating system chooses the
+smallest non-negative integer that does not yet correspond to an open file
+descriptor.  So, when a given fd (0, 1, or 2) is closed, opening a new file
+descriptor may assign the new file descriptor to this fd.  This can have
+unintended effects, because now standard input/output/error of your process
+is referring to a file that was not meant to be used in that role.
+
+This situation is a security risk because the behaviour of the program
+in this situation was surely never tested, therefore anything can happen
+then -- from overwriting precious files of the user to endless loops.
+
+To deal with this situation, you first need to determine whether your
+program is affected by the problem.
address@hidden @bullet
address@hidden
+Does your program invoke functions that allocate new file descriptors?
+These are the system calls
address@hidden @bullet
address@hidden
address@hidden()}, @code{openat()}, @code{creat()}
address@hidden
address@hidden()}
address@hidden
address@hidden()}, @code{freopen()}
address@hidden
address@hidden()}, @code{pipe2()}, @code{popen()}
address@hidden
address@hidden()}
address@hidden
address@hidden()}, @code{mkstemp()}, @code{mkstemps()}, @code{mkostemp()},
address@hidden()}
address@hidden itemize
address@hidden
+Note that you also have to consider the libraries that your program uses.
address@hidden
+If your program may open two or more file descriptors or FILE streams for
+reading at the same time, and some of them may reference standard input,
+your program @emph{is affected}.
address@hidden
+If your program may open two or more file descriptors or FILE streams for
+writing at the same time, and some of them may reference standard output
+or standard error, your program @emph{is affected}.
address@hidden
+If your program does not open new file descriptors or FILE streams, it is
address@hidden affected}.
address@hidden
+If your program opens only one new file descriptor or FILE stream at a time,
+it is @emph{not affected}.  This is often the case for programs that are
+structured in simple phases: first a phase where input is read from a file
+into memory, then a phase of processing in memory, finally a phase where
+the result is written to a file.
address@hidden
+If your program opens only two new file descriptors or FILE streams at a
+time, out of which one is for reading and the one is for writing, it is
address@hidden affected}.  This is because if the first file descriptor is
+allocated and the second file descriptor is picked as 0, 1, or 2, and
+both happen to be the same, writing to the one opened in @code{O_RDONLY}
+mode will produce an error @code{EBADF}, as desired.
address@hidden itemize
+
+If your program is affected, what is the mitigation?
+
+Some operating systems install open file descriptors in place of the
+closed ones, either in the @code{exec} system call or during program
+startup.  When such a file descriptor is accessed through a system call,
+it behaves like an open file descriptor opened for the ``wrong'' direction:
+the system calls @code{fcntl()} and @code{fstat()} succeed, whereas
address@hidden()} from fd 0 and @code{write()} to fd 1 or 2 fail with error
address@hidden ("Bad file descriptor").  The important point here is that
+when your program allocates a new file descriptor, it will have a value
+greater than 2.
+
+This mitigation is enabled on HP-UX, for all programs, and on glibc,
+FreeBSD, NetBSD, OpenBSD, but only for setuid or setgid programs.  Since
+it is operating system dependent, it is not a complete mitigation.
+
+For a complete mitigation, Gnulib provides two alternative sets of modules:
address@hidden @bullet
address@hidden
+The @code{xstdopen} module.
address@hidden
+The @code{*-safer} modules:
address@hidden,
address@hidden,
address@hidden,
address@hidden,
address@hidden,
address@hidden,
address@hidden,
address@hidden,
address@hidden,
address@hidden
address@hidden itemize
+
+The approach with the @code{xstdopen} module is simpler, but it adds three
+system calls to program startup.  Whereas the approach with the @code{*-safer}
+modules is more complex, but adds no overhead (no additional system calls)
+in the normal case.
+
+To use the approach with the @code{xstdopen} module:
address@hidden
address@hidden
+Import the module @code{xstdopen} from Gnulib.
address@hidden
+In the compilation unit that contains the @code{main} function, include
address@hidden"xstdopen.h"}.
address@hidden
+In the @code{main} function, near the beginning, namely right after
+the i18n related initializations (@code{setlocale}, @code{bindtextdomain},
address@hidden invocations, if any) and
+the @code{closeout} initialization (if any), insert the invocation:
address@hidden
+/* Ensure that stdin, stdout, stderr are open.  */
+xstdopen ();
address@hidden smallexample
address@hidden enumerate
+
+To use the approach with the @code{*-safer} modules:
address@hidden
address@hidden
+Import the relevant modules from Gnulib.
address@hidden
+In the compilation units that contain these function calls, include the
+replacement header file.
address@hidden enumerate
+Do so according to this table:
address@hidden @columnfractions .28 .32 .4
address@hidden Function @tab Module @tab Header file
address@hidden @code{open()}
address@hidden @code{fcntl-safer}
address@hidden @code{"fcntl--.h"}
address@hidden @code{openat()}
address@hidden @code{openat-safer}
address@hidden @code{"fcntl--.h"}
address@hidden @code{creat()}
address@hidden @code{fcntl-safer}
address@hidden @code{"fcntl--.h"}
address@hidden @code{dup()}
address@hidden @code{unistd-safer}
address@hidden @code{"unistd--.h"}
address@hidden @code{fopen()}
address@hidden @code{fopen-safer}
address@hidden @code{"stdio--.h"}
address@hidden @code{freopen()}
address@hidden @code{freopen-safer}
address@hidden @code{"stdio--.h"}
address@hidden @code{pipe()}
address@hidden @code{unistd-safer}
address@hidden @code{"unistd--.h"}
address@hidden @code{pipe2()}
address@hidden @code{pipe2-safer}
address@hidden @code{"unistd--.h"}
address@hidden @code{popen()}
address@hidden @code{popen-safer}
address@hidden @code{"stdio--.h"}
address@hidden @code{opendir()}
address@hidden @code{dirent-safer}
address@hidden @code{"dirent--.h"}
address@hidden @code{tmpfile()}
address@hidden @code{tmpfile-safer}
address@hidden @code{"stdio--.h"}
address@hidden @code{mkstemp()}
address@hidden @code{stdlib-safer}
address@hidden @code{"stdlib--.h"}
address@hidden @code{mkstemps()}
address@hidden @code{stdlib-safer}
address@hidden @code{"stdlib--.h"}
address@hidden @code{mkostemp()}
address@hidden @code{stdlib-safer}
address@hidden @code{"stdlib--.h"}
address@hidden @code{mkostemps()}
address@hidden @code{stdlib-safer}
address@hidden @code{"stdlib--.h"}
address@hidden multitable