Re: [Qemu-block] Disconecting /dev/nbdX leaves stale partitions and device

[Stefan Hajnoczi]

On Thu, Jul 19, 2018 at 01:56:34PM +0300, Nir Soffer wrote:
> On Wed, Jul 18, 2018 at 6:35 PM Stefan Hajnoczi <address@hidden> wrote:
> 
> > On Wed, Jul 11, 2018 at 08:28:38PM +0300, Nir Soffer wrote:
> > > On Wed, Jul 11, 2018 at 3:40 PM Stefan Hajnoczi <address@hidden>
> > wrote:
> > >
> > > > On Thu, Jul 05, 2018 at 02:10:58AM +0300, Nir Soffer wrote:
> > > >
> > > > CCing Eric and Paolo, who maintain QEMU's NBD code.
> > > >
> > > > > I'm trying to access a qcow2 image via /dev/nbdX device.
> > > >
> > > > Untrusted disk images should not be attached to the host using qemu-nbd
> > > > (or loopback devices) for security reasons.  If you hit this bug during
> > > > development or ad-hoc qemu-nbd usage, then that's fine.  But I just
> > > > wanted to post a reminder that production use cases should not use this
> > > > feature when dealing with untrusted disk images.
> > > >
> > >
> > > Thanks for the warning. We intend to use it with oVirt disks, which are
> > > generally
> > > created by qemu. But these disks may also be uploaded by the oVirt
> > > administrator
> > > or by a backup application. They can also be modified by anything
> > running on
> > > a hypervisor, since oVirt storage is available on all hypervisors.
> > Finally
> > > it can be
> > > corrupted on storage.
> > >
> > > Basically anything we feed to qemu or qemu-img may be fed to qemu-nbd.
> > > We assume that qemu-nbd is robust to handle broken images.
> >
> > qemu-nbd doesn't post a security risk since it does not interpret the
> > contents of the disk.  (Assuming oVirt explicitly specifies the image
> > file format using qemu-nbd --format=FORMAT.  This is important, too.)
> >
> 
> Sure, we always specify the format.
> 
> 
> > The security risk is attaching an untrusted disk to the host via
> > /dev/nbdX.  The Linux developers do not consider this secure.
> 
> 
> Is this a specific issue with nbd, or the general issue of attaching a
> block device
> with untrusted content?

This issue applies to any block device with untrusted contents.

> > For
> > example, mounting a filesystem from an untrusted disk could compromise
> > the host.
> 
> 
> We don't mount  anything, we just need access to the guest data, so we can
> stream it to clients over HTTPS.
> 
> 
> > There are other vectors like partition table probing or
> > userspace activity in response to the disk add uevent (udev).
> >
> 
> Isn't this the same issue with any LUN or LVM logical volume?
> 
> We already attach LUNs with guest data to hosts, or LVM logical voluems
> with guest data. I guess both are considered as untrusted since the guest
> control the content of the block device.

Yes.  It's generally best to avoid doing this.

It's not a black-and-white security issue.  For example, lvm.conf allows
you to filter specific devices that will be scanned for physical
volumes.  This way you avoid scanning untrusted USB sticks, NBD volumes,
etc.  But this doesn't scale well: there are other components in the
system that might also interpret the contents of the device, so are you
100% sure there is no security hole?

> > This is why libguestfs always accesses disks from an appliance VM
> > instead of attaching them directly on the host.
> >
> > > Without this we
> > > would not be able to provide guest data for incremental backup or other
> > > purposes.
> >
> > You can provide access to the data without attaching it as a host block
> > device (/dev/nbdX).
> >
> > Drop the -c argument and use --socket=PATH (UNIX domain socket) or
> > --port=TCP_PORT (TCP listen socket) instead.  Now you have an NBD server
> > that accesses the image file from userspace (without attaching a host
> > block device).  This avoids the security issues with host block devices.
> >
> 
> This looks like a better solution, avoiding root access to attach to
> /dev/nbdX,
> but this makes nbd less attractive for oVirt because we don't have nbd
> client
> supporting streaming.
> 
> We are missing libnbd, allowing reading and writing to nbd-server. Can we
> extract code from qemu-img to create such library? Having libnbd with
> bindings
> to popular languages can make nbd much attractive solution for the
> community.
>
> I know Richard is working on adding nbd support for libcurl, but I'm not
> sure
> curl is the best place to integrate it.
> https://curl.haxx.se/mail/lib-2018-07/0052.html
> 
> Basically we we need is something like:
> 
>     nbd_map(fd) - return list of allocated ranges
>     nbd_pread(fd, offset, size, buf) - read data in range into buf
>     nbd_pwrite(fd, offset, size, buf) - write data size bytes in but to
> offset
> 
> (The library api should be probably be bigger, matching the procotol, but
> I'm ignoring anything we don't need now)
> 
> With this we can easily expose HTTP API to access the data.
> 
>     GET /resource/map
> 
> return output similar to qemu-img map --output json
> 
>     GET /resource
>     Range: bytes=START-END
> 
> Returns the guest data in the byte range. We already have this:
> http://ovirt.github.io/ovirt-imageio/random-io.html)
> 
> Having HTTP API makes it easy to integrate with. This is also the protocol
> Kubrvirt CDI[1] is using for importing images:
> https://github.com/kubevirt/containerized-data-importer

2 options:

1. Write a server that does HTTP<->NBD.  Less efficient but fairly easy
   to do in Go, Python, etc.

2. Add the HTTP server to QEMU itself (similar to the nbd-server-start
   QMP command).  This is efficient but HTTP(S) is non-trivial nowadays,
   so I'm not sure if adding an HTTP(S) server to QEMU is a good idea.

The advantage I see in #2 is that HTTP is the universal network
protocol and I'm sure there would be many new use cases for this
interface.

Stefan

signature.asc
Description: PGP signature