Re: [Qemu-devel] [V4 PATCH 2/8] Provide chroot environment server side interfaces

[Daniel P. Berrange]

On Tue, Feb 01, 2011 at 10:55:39AM +0530, M. Mohan Kumar wrote:
> Implement chroot server side interfaces like sending the file
> descriptor to qemu process, reading the object request from socket etc.
> Also add chroot main function and other helper routines.
> 
> Signed-off-by: M. Mohan Kumar <address@hidden>
> ---
>  Makefile.objs              |    1 +
>  hw/9pfs/virtio-9p-chroot.c |  212 
> ++++++++++++++++++++++++++++++++++++++++++++
>  hw/9pfs/virtio-9p-chroot.h |   41 +++++++++
>  hw/9pfs/virtio-9p.c        |   33 +++++++
>  hw/file-op-9p.h            |    3 +
>  5 files changed, 290 insertions(+), 0 deletions(-)
>  create mode 100644 hw/9pfs/virtio-9p-chroot.c
>  create mode 100644 hw/9pfs/virtio-9p-chroot.h
> 
> diff --git a/Makefile.objs b/Makefile.objs
> index bc0344c..3007b6d 100644
> --- a/Makefile.objs
> +++ b/Makefile.objs
> +/*
> + * Fork a process and chroot into the share path. Communication
> + * between qemu process and chroot process happens via socket
> + * All file descriptors (including stdout and stderr) are closed
> + * except one socket descriptor (which is used for communicating
> + * between qemu process and chroot process)
> + */
> +int v9fs_chroot(FsContext *fs_ctx)
> +{
> +    int fd_pair[2], chroot_sock, error;
> +    V9fsFileObjectRequest request;
> +    pid_t pid;
> +    uint64_t code;
> +    FdInfo fd_info;
> +
> +    if (socketpair(PF_UNIX, SOCK_STREAM, 0, fd_pair) < 0) {
> +        error_report("socketpair %s", strerror(errno));
> +        return -1;
> +    }
> +
> +    pid = fork();
> +    if (pid < 0) {
> +        error_report("fork %s", strerror(errno));
> +        return -1;
> +    }
> +    if (pid != 0) {
> +        fs_ctx->chroot_socket = fd_pair[0];
> +        close(fd_pair[1]);
> +        return 0;
> +    }
> +
> +    close(fd_pair[0]);
> +    chroot_sock = fd_pair[1];
> +    if (chroot(fs_ctx->fs_root) < 0) {
> +        code = CHROOT_ERROR << 32 | errno;
> +        error = qemu_write_full(chroot_sock, &code, sizeof(code));
> +        _exit(1);
> +    }
> +
> +    error = chroot_daemonize(chroot_sock);
> +    if (error) {
> +        code = SETSID_ERROR << 32 | error;
> +        error = qemu_write_full(chroot_sock, &code, sizeof(code));
> +        _exit(1);
> +    }
> +
> +    /*
> +     * Write 0 to chroot socket to indicate chroot process creation is
> +     * successful
> +     */
> +    code = 0;
> +    if (qemu_write_full(chroot_sock, &code, sizeof(code))
> +                    != sizeof(code)) {
> +        _exit(1);
> +    }
> +    /* get the request from the socket */
> +    while (1) {
> +        memset(&fd_info, 0, sizeof(fd_info));
> +        if (chroot_read_request(chroot_sock, &request) == EIO) {
> +            fd_info.fi_fd = 0;
> +            fd_info.fi_error = EIO;
> +            fd_info.fi_flags = FI_SOCKERR;
> +            chroot_sendfd(chroot_sock, &fd_info);
> +            continue;
> +        }
> +        qemu_free((void *)request.path.path);
> +        if (request.data.oldpath_len) {
> +            qemu_free((void *)request.path.old_path);
> +        }
> +    }
> +}

There is a subtle problem with using fork() in a multi-threaded
program that I was recently made aware of in libvirt. In short
if you have a multi-threaded program that calls fork(), then
the child process must only use POSIX functions that are
declared 'async signal safe', until the child calls exec() or
exit().  In particular any malloc()/free() related functions
are *not* async signal safe.

  http://pubs.opengroup.org/onlinepubs/9699919799/functions/fork.html

  "If a multi-threaded process calls fork(), the new process shall contain
  a replica of the calling thread and its entire address space, possibly
  including the states of mutexes and other resources. Consequently, to
  avoid errors, the child process may only execute async-signal-safe
  operations until such time as one of the exec functions is called."

One example problem scenario. Thread 1 is currently doing a
malloc() and the malloc() impl is holding a mutex. Thread 2
now does a fork(), and in the child process calls malloc().
The child process will deadlock / hang forever because there
is nothing which will ever release the malloc() mutex that
was originally held by Thread 1. See also this thread which
brought the problem to my attention:

  http://lists.gnu.org/archive/html/coreutils/2011-01/msg00085.html

Regards,
Daniel