[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2 000/109] virtiofs daemon [all]

From: Philippe Mathieu-Daudé
Subject: Re: [PATCH v2 000/109] virtiofs daemon [all]
Date: Tue, 21 Jan 2020 16:41:40 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2

On 1/21/20 1:22 PM, Dr. David Alan Gilbert (git) wrote:
From: "Dr. David Alan Gilbert" <address@hidden>

   This is a full set for virtiofsd - a daemon
that implements the user space side of virtiofs.

I've addressed most review comments from V1; we've got
95 Reviewed-by's out of 110 - I'd like to get a PULL out ASAP.

The set pulls in a big chunk of the upstream libfuse library
(unmodified so that it's easy to check it really is upstream),
chops all the stuff out we don't need and then adds the
new transport we need.  I've formatted everything into qemu's code style -
using indent and the clang tools for the files I've imported.

We can't just link with libfuse, since we have to make ABI incompatible
changes for the new transport and it's quite invasive; the library is
designed to be the basis for multiple filesystems, but all on the same

Running this daemon is typically done with:

    ./virtiofsd --socket-path=/path/socket -o source=/path/to/fs

connected to a qemu that's then started with:
    -chardev socket,id=char0,path=/path/socket -device 

and then in the guest mount with:
    mount -t virtiofs myfs /mnt

Our development branch is: https://gitlab.com/virtio-fs/qemu/tree/virtio-fs-dev

FWIW this branch contains more than this series, because it failed to build on 32-bit:

$ make virtiofsd
  CC      tools/virtiofsd/fuse_lowlevel.o
tools/virtiofsd/fuse_lowlevel.c: In function 'do_removemapping':
tools/virtiofsd/fuse_lowlevel.c:1956:60: error: format '%ld' expects argument of type 'long int', but argument 4 has type 'unsigned int' [-Werror=format=] "do_removemapping: invalid in, expected %d * %ld, has %ld - %ld\n",
tools/virtiofsd/fuse_lowlevel.c:1956:69: error: format '%ld' expects argument of type 'long int', but argument 5 has type 'size_t {aka unsigned int}' [-Werror=format=] "do_removemapping: invalid in, expected %d * %ld, has %ld - %ld\n",
tools/virtiofsd/fuse_lowlevel.c:1956:75: error: format '%ld' expects argument of type 'long int', but argument 6 has type 'size_t {aka unsigned int}' [-Werror=format=] "do_removemapping: invalid in, expected %d * %ld, has %ld - %ld\n",


And the offending patch is not included in the 109:

$ git show 62d82cfbe5a
commit 62d82cfbe5a2ad05ae3215f2461b76aa9915108d
Author: Dr. David Alan Gilbert <address@hidden>
Date:   Fri Jul 6 18:03:49 2018 +0100

    DAX: virtiofsd: Add setup/remove mappings fuse commands

Consider pushing tags with version, so it is easier to diff (virtio-fs-dev got force-pushed).

We don't need libslirp/capstone/dtc but they are still built:

$ make virtiofsd
  GEN     config-host.h
make[1]: Entering directory 'slirp'
  CC      slirp/src/state.o
  CC      slirp/src/tcp_timer.o
  CC      slirp/src/dhcpv6.o
  CC      slirp/src/ip_input.o
  CC      slirp/src/ip6_icmp.o
  CC      slirp/src/bootp.o
  CC      slirp/src/ip6_input.o
  CC      slirp/src/slirp.o
  CC      slirp/src/vmstate.o
  CC      slirp/src/ip_output.o
  CC      slirp/src/ncsi.o
  CC      slirp/src/tcp_output.o
  CC      slirp/src/ndp_table.o
  CC      slirp/src/version.o
  CC      slirp/src/misc.o
  CC      slirp/src/ip6_output.o
  CC      slirp/src/mbuf.o
  CC      slirp/src/tftp.o
  CC      slirp/src/arp_table.o
  CC      slirp/src/util.o
  CC      slirp/src/socket.o
  CC      slirp/src/sbuf.o
  CC      slirp/src/stream.o
  CC      slirp/src/dnssearch.o
  CC      slirp/src/udp.o
  CC      slirp/src/tcp_input.o
  CC      slirp/src/if.o
  CC      slirp/src/cksum.o
  CC      slirp/src/tcp_subr.o
  CC      slirp/src/udp6.o
  CC      slirp/src/ip_icmp.o
  AR      slirp/libslirp.a
make[1]: Leaving directory 'slirp'
         DEP dtc/libfdt/fdt_overlay.c
         DEP dtc/libfdt/fdt_addresses.c
         DEP dtc/libfdt/fdt_empty_tree.c
         DEP dtc/libfdt/fdt_strerror.c
         DEP dtc/libfdt/fdt_rw.c
         DEP dtc/libfdt/fdt_sw.c
         DEP dtc/libfdt/fdt_wip.c
         DEP dtc/libfdt/fdt_ro.c
         DEP dtc/libfdt/fdt.c
         DEP dtc/util.c
         DEP dtc/fdtoverlay.c
         DEP dtc/fdtput.c
         DEP dtc/fdtget.c
         DEP dtc/fdtdump.c
         DEP convert-dtsv0-lexer.lex.c
         DEP dtc/srcpos.c
         BISON dtc-parser.tab.c
         DEP dtc-parser.tab.c
         LEX dtc-lexer.lex.c
         DEP dtc-lexer.lex.c
         DEP dtc/treesource.c
         DEP dtc/livetree.c
         DEP dtc/fstree.c
         DEP dtc/flattree.c
         DEP dtc/dtc.c
         DEP dtc/data.c
         DEP dtc/checks.c
         CC libfdt/fdt.o
         CC libfdt/fdt_ro.o
         CC libfdt/fdt_wip.o
         CC libfdt/fdt_sw.o
         CC libfdt/fdt_rw.o
         CC libfdt/fdt_strerror.o
         CC libfdt/fdt_empty_tree.o
         CC libfdt/fdt_addresses.o
         CC libfdt/fdt_overlay.o
         AR libfdt/libfdt.a
arm-linux-gnueabihf-ar: creating libfdt/libfdt.a
a - libfdt/fdt.o
a - libfdt/fdt_ro.o
a - libfdt/fdt_wip.o
a - libfdt/fdt_sw.o
a - libfdt/fdt_rw.o
a - libfdt/fdt_strerror.o
a - libfdt/fdt_empty_tree.o
a - libfdt/fdt_addresses.o
a - libfdt/fdt_overlay.o
  CC      cs.o
  CC      utils.o
  CC      SStream.o
  CC      MCInstrDesc.o
  CC      MCRegisterInfo.o
  CC      arch/ARM/ARMDisassembler.o
  CC      arch/ARM/ARMInstPrinter.o
  CC      arch/ARM/ARMMapping.o
  CC      arch/ARM/ARMModule.o
  CC      arch/AArch64/AArch64BaseInfo.o
  CC      arch/AArch64/AArch64Disassembler.o
  CC      arch/AArch64/AArch64InstPrinter.o
  CC      arch/AArch64/AArch64Mapping.o
  CC      arch/AArch64/AArch64Module.o
  CC      arch/Mips/MipsDisassembler.o
  CC      arch/Mips/MipsInstPrinter.o
  CC      arch/Mips/MipsMapping.o
  CC      arch/Mips/MipsModule.o
  CC      arch/PowerPC/PPCDisassembler.o
  CC      arch/PowerPC/PPCInstPrinter.o
  CC      arch/PowerPC/PPCMapping.o
  CC      arch/PowerPC/PPCModule.o
  CC      arch/Sparc/SparcDisassembler.o
  CC      arch/Sparc/SparcInstPrinter.o
  CC      arch/Sparc/SparcMapping.o
  CC      arch/Sparc/SparcModule.o
  CC      arch/SystemZ/SystemZDisassembler.o
  CC      arch/SystemZ/SystemZInstPrinter.o
  CC      arch/SystemZ/SystemZMapping.o
  CC      arch/SystemZ/SystemZModule.o
  CC      arch/SystemZ/SystemZMCTargetDesc.o
  CC      arch/X86/X86DisassemblerDecoder.o
  CC      arch/X86/X86Disassembler.o
  CC      arch/X86/X86IntelInstPrinter.o
  CC      arch/X86/X86ATTInstPrinter.o
  CC      arch/X86/X86Mapping.o
  CC      arch/X86/X86Module.o
  CC      arch/XCore/XCoreDisassembler.o
  CC      arch/XCore/XCoreInstPrinter.o
  CC      arch/XCore/XCoreMapping.o
  CC      arch/XCore/XCoreModule.o
  CC      MCInst.o
  AR      libcapstone.a
  CC      tools/virtiofsd/buffer.o
  CC      tools/virtiofsd/fuse_opt.o
  CC      tools/virtiofsd/fuse_log.o
  CC      tools/virtiofsd/fuse_lowlevel.o

FYI Paolo queued this patch, so libfdt shouldn't get compiled anymore:

This code is going into tools/virtiofsd  based on the previous long

Thank you to all those who have contributed code and to those who have
reviewed the previous version.


Dr. David Alan Gilbert (35):
   virtiofsd: Pull in upstream headers
   virtiofsd: Pull in kernel's fuse.h
   virtiofsd: Add auxiliary .c's
   virtiofsd: Add fuse_lowlevel.c
   virtiofsd: Add passthrough_ll
   virtiofsd: Trim down imported files
   virtiofsd: Format imported files to qemu style
   virtiofsd: Fix fuse_daemonize ignored return values
   virtiofsd: Fix common header and define for QEMU builds
   virtiofsd: Trim out compatibility code
   virtiofsd: Add options for virtio
   virtiofsd: Open vhost connection instead of mounting
   virtiofsd: Start wiring up vhost-user
   virtiofsd: Add main virtio loop
   virtiofsd: get/set features callbacks
   virtiofsd: Start queue threads
   virtiofsd: Poll kick_fd for queue
   virtiofsd: Start reading commands from queue
   virtiofsd: Send replies to messages
   virtiofsd: Keep track of replies
   virtiofsd: Add Makefile wiring for virtiofsd contrib
   virtiofsd: Fast path for virtio read
   virtiofs: Add maintainers entry
   virtiofsd: Plumb fuse_bufvec through to do_write_buf
   virtiofsd: Pass write iov's all the way through
   virtiofsd: cap-ng helpers
   docs:  Add docs/tools
   virtiofsd: Handle reinit
   virtiofsd: Handle hard reboot
   virtiofsd: Kill threads when queues are stopped
   vhost-user: Print unexpected slave message types
   contrib/libvhost-user: Protect slave fd with mutex
   virtiofsd: Clean up inodes on destroy
   libvhost-user: Fix some memtable remap cases
   virtiofsd: Convert lo_destroy to take the lo->mutex lock itself

Eric Ren (1):
   virtiofsd: fix incorrect error handling in lo_do_lookup

Eryu Guan (3):
   virtiofsd: print log only when priority is high enough
   virtiofsd: convert more fprintf and perror to use fuse log infra
   virtiofsd: stop all queue threads on exit in virtio_loop()

Jiufei Xue (1):
   virtiofsd: support nanosecond resolution for file timestamp

Liu Bo (6):
   virtiofsd: fix error handling in main()
   virtiofsd: cleanup allocated resource in se
   virtiofsd: fix memory leak on lo.source
   virtiofsd: add helper for lo_data cleanup
   virtiofsd: enable PARALLEL_DIROPS during INIT
   Virtiofsd: fix memory leak on fuse queueinfo

Masayoshi Mizuma (4):
   virtiofsd: Add ID to the log with FUSE_LOG_DEBUG level
   virtiofsd: Add timestamp to the log with FUSE_LOG_DEBUG level
   virtiofsd: Prevent multiply running with same vhost_user_socket
   virtiofsd: add some options to the help message

Miklos Szeredi (10):
   virtiofsd: passthrough_ll: add fallback for racy ops
   virtiofsd: passthrough_ll: add renameat2 support
   virtiofsd: passthrough_ll: disable readdirplus on cache=never
   virtiofsd: passthrough_ll: control readdirplus
   virtiofsd: rename unref_inode() to unref_inode_lolocked()
   virtiofsd: fail when parent inode isn't known in lo_do_lookup()
   virtiofsd: extract root inode init into setup_root()
   virtiofsd: passthrough_ll: clean up cache related options
   virtiofsd: passthrough_ll: use hashtable
   virtiofsd: passthrough_ll: fix refcounting on remove/rename

Misono Tomohiro (1):
   virtiofsd: Fix data corruption with O_APPEND write in writeback mode

Peng Tao (1):
   virtiofsd: do not always set FUSE_FLOCK_LOCKS

Stefan Hajnoczi (37):
   virtiofsd: remove mountpoint dummy argument
   virtiofsd: remove unused notify reply support
   virtiofsd: add -o source=PATH to help output
   virtiofsd: add --fd=FDNUM fd passing option
   virtiofsd: make -f (foreground) the default
   virtiofsd: add vhost-user.json file
   virtiofsd: add --print-capabilities option
   virtiofsd: passthrough_ll: add lo_map for ino/fh indirection
   virtiofsd: passthrough_ll: add ino_map to hide lo_inode pointers
   virtiofsd: passthrough_ll: add dirp_map to hide lo_dirp pointers
   virtiofsd: passthrough_ll: add fd_map to hide file descriptors
   virtiofsd: validate path components
   virtiofsd: add fuse_mbuf_iter API
   virtiofsd: validate input buffer sizes in do_write_buf()
   virtiofsd: check input buffer size in fuse_lowlevel.c ops
   virtiofsd: prevent ".." escape in lo_do_lookup()
   virtiofsd: prevent ".." escape in lo_do_readdir()
   virtiofsd: use /proc/self/fd/ O_PATH file descriptor
   virtiofsd: sandbox mount namespace
   virtiofsd: move to an empty network namespace
   virtiofsd: move to a new pid namespace
   virtiofsd: add seccomp whitelist
   virtiofsd: set maximum RLIMIT_NOFILE limit
   virtiofsd: fix libfuse information leaks
   virtiofsd: add security guide document
   virtiofsd: add --syslog command-line option
   virtiofsd: use fuse_lowlevel_is_virtio() in fuse_session_destroy()
   virtiofsd: prevent fv_queue_thread() vs virtio_loop() races
   virtiofsd: make lo_release() atomic
   virtiofsd: prevent races with lo_dirp_put()
   virtiofsd: rename inode->refcount to inode->nlookup
   virtiofsd: add man page
   virtiofsd: introduce inode refcount to prevent use-after-free
   virtiofsd: process requests in a thread pool
   virtiofsd: prevent FUSE_INIT/FUSE_DESTROY races
   virtiofsd: fix lo_destroy() resource leaks
   virtiofsd: add --thread-pool-size=NUM option

Vivek Goyal (6):
   virtiofsd: Make fsync work even if only inode is passed in
   virtiofsd: passthrough_ll: create new files in caller's context
   virtiofsd: Parse flag FUSE_WRITE_KILL_PRIV
   virtiofsd: Drop CAP_FSETID if client asked for it
   virtiofsd: Support remote posix locks
   virtiofsd: Reset O_DIRECT flag during file open

Xiao Yang (2):
   vitriofsd/passthrough_ll: fix fallocate() ifdefs
   virtiofsd/passthrough_ll: Pass errno to fuse_reply_err()

piaojun (2):
   virtiofsd: add definition of fuse_buf_writev()
   virtiofsd: use fuse_buf_writev to replace fuse_buf_write for better

  .gitignore                                |    1 +
  MAINTAINERS                               |    8 +
  Makefile                                  |   27 +-
  Makefile.objs                             |    1 +
  configure                                 |   16 +
  contrib/libvhost-user/libvhost-user.c     |   57 +-
  contrib/libvhost-user/libvhost-user.h     |    6 +
  docs/index.rst                            |    1 +
  docs/interop/vhost-user.json              |    4 +-
  docs/tools/conf.py                        |   16 +
  docs/tools/index.rst                      |   14 +
  docs/tools/virtiofsd-security.rst         |  118 +
  hw/virtio/vhost-user.c                    |    2 +-
  include/standard-headers/linux/fuse.h     |  891 ++++++
  scripts/update-linux-headers.sh           |    1 +
  tools/virtiofsd/50-qemu-virtiofsd.json.in |    5 +
  tools/virtiofsd/Makefile.objs             |   12 +
  tools/virtiofsd/buffer.c                  |  351 +++
  tools/virtiofsd/fuse.h                    | 1249 +++++++++
  tools/virtiofsd/fuse_common.h             |  860 ++++++
  tools/virtiofsd/fuse_i.h                  |  115 +
  tools/virtiofsd/fuse_log.c                |   40 +
  tools/virtiofsd/fuse_log.h                |   74 +
  tools/virtiofsd/fuse_lowlevel.c           | 2779 +++++++++++++++++++
  tools/virtiofsd/fuse_lowlevel.h           | 2023 ++++++++++++++
  tools/virtiofsd/fuse_misc.h               |   60 +
  tools/virtiofsd/fuse_opt.c                |  449 +++
  tools/virtiofsd/fuse_opt.h                |  272 ++
  tools/virtiofsd/fuse_signals.c            |   97 +
  tools/virtiofsd/fuse_virtio.c             |  985 +++++++
  tools/virtiofsd/fuse_virtio.h             |   33 +
  tools/virtiofsd/helper.c                  |  349 +++
  tools/virtiofsd/passthrough_helpers.h     |   51 +
  tools/virtiofsd/passthrough_ll.c          | 3006 +++++++++++++++++++++
  tools/virtiofsd/seccomp.c                 |  164 ++
  tools/virtiofsd/seccomp.h                 |   16 +
  tools/virtiofsd/virtiofsd.texi            |  104 +
  37 files changed, 14242 insertions(+), 15 deletions(-)
  create mode 100644 docs/tools/conf.py
  create mode 100644 docs/tools/index.rst
  create mode 100644 docs/tools/virtiofsd-security.rst
  create mode 100644 include/standard-headers/linux/fuse.h
  create mode 100644 tools/virtiofsd/50-qemu-virtiofsd.json.in
  create mode 100644 tools/virtiofsd/Makefile.objs
  create mode 100644 tools/virtiofsd/buffer.c
  create mode 100644 tools/virtiofsd/fuse.h
  create mode 100644 tools/virtiofsd/fuse_common.h
  create mode 100644 tools/virtiofsd/fuse_i.h
  create mode 100644 tools/virtiofsd/fuse_log.c
  create mode 100644 tools/virtiofsd/fuse_log.h
  create mode 100644 tools/virtiofsd/fuse_lowlevel.c
  create mode 100644 tools/virtiofsd/fuse_lowlevel.h
  create mode 100644 tools/virtiofsd/fuse_misc.h
  create mode 100644 tools/virtiofsd/fuse_opt.c
  create mode 100644 tools/virtiofsd/fuse_opt.h
  create mode 100644 tools/virtiofsd/fuse_signals.c
  create mode 100644 tools/virtiofsd/fuse_virtio.c
  create mode 100644 tools/virtiofsd/fuse_virtio.h
  create mode 100644 tools/virtiofsd/helper.c
  create mode 100644 tools/virtiofsd/passthrough_helpers.h
  create mode 100644 tools/virtiofsd/passthrough_ll.c
  create mode 100644 tools/virtiofsd/seccomp.c
  create mode 100644 tools/virtiofsd/seccomp.h
  create mode 100644 tools/virtiofsd/virtiofsd.texi

reply via email to

[Prev in Thread] Current Thread [Next in Thread]