Re: Exporting qcow2 images as raw data from ova file with qemu-nbd

[Nir Soffer]

On Tue, Jun 23, 2020 at 1:47 PM Richard W.M. Jones <rjones@redhat.com> wrote:
>
> On Tue, Jun 23, 2020 at 01:14:43PM +0300, Nir Soffer wrote:
> > On Tue, Jun 23, 2020 at 12:47 PM Richard W.M. Jones <rjones@redhat.com> 
> > wrote:
> > > Here you go:
> > >
> > > https://github.com/libguestfs/nbdkit/commit/2d15e79f65764d9b0c68bea28ed6afbcbcc63467
> >
> > Nice!
> >
> > But using qemu-nbd directly is much simpler and will perform better.
>
> Not sure about simpler,

These are the patches (in review) implementing this in imageio client:

- https://gerrit.ovirt.org/c/109847
- https://gerrit.ovirt.org/c/109848

And this is example script for the engine SDK:
https://gerrit.ovirt.org/c/109873/

Work is not finished yet, we need to handle getting image virtual size
and measuring
required size to support upload to sparse disks on block storage:
https://bugzilla.redhat.com/show_bug.cgi?id=1849981#c3

Since we already run qemu-nbd to access images, this is just a
modification of the
'json:{...}' filename, so we can also consume images inside tar file.

And in the client, allow specifying image format and tar member name
(like the tar plugin).

Using tar plugin requires:
- Adding support for NBD url - this is planned but at least the same
amount of work as supporting
  json: with qcow2 drive over raw with range.
- Integrating with nbdkit and qemu-nbd (since we must have qcow2 support)
- packaging
- handle missing dependencies

Whats's missing in the current solution is supporting compressed raw
disk. I don't think
oVirt creates such ova files, so this should not be an issue. However
if nbdkit tar
plugin can support this (current implementation does not), this will
be a good reason
to integrate with it.

In this case we will have:

    [ovf | gziped raw image | ...]  -> nbdkit exposing uncompressed
raw image -> imageio nbd client

I wonder if it is possible to  add gzip driver in qemu block layer?

> and you might want to verify the "perform better" claim

I did not test yet on real server and harder, but from initial tests I
get same performance
from uploading qcow2 compressed image and uploading qcow2 compressed
image inside
ova file.

> (it is likely to be true because writing a plugin in
> Python causes requests to be serialized, but it may not matter if
> you're reading linearly from a file).

It matters, because having serial reads also imply serial writes to the server
in imageio. We could change this but the current design we have multiple threads
reading from nbd server, and sending data or zero requests to imageio server.
This doubles the performance we had in ovirt 4.3.

You can see here results compared to qemu-img convert from local image
to block device:
https://github.com/oVirt/ovirt-imageio/commit/97f2e277458db579023ba54a4a4bd122b36f543e

The unexpected slow results with qemu-img are caused by the pre-zero
pessimisation
which should be removed soon:
https://lists.nongnu.org/archive/html/qemu-block/2020-06/msg01094.html

With the slow pre-zeroing removed, qemu-img convert gives similar performance.

> The tar plugin could be rewritten in C if performance was really a problem.

This may make the tar plugin more attractive, especially if we can
support multiple
readers. Maybe we can separate the tar parsing to a helper process, and use the
results (list of (member, offset, size)) in a file-like plugin in C?


> > Regardless, nbdit tar plugin is awesome. Is it possible to expose
> > all the disks from a tar file so they are accessible using the
> > export name?
>
> In theory yes, but it would require exposing the export name
> (nbdkit_export_name() -> nbdkit.export_name()) to Python plugins,
> which we don't do at the moment.  See plugins/python/python.c:
> NbdkitMethods[].  That would also mean the plugin would require the
> latest nbdkit so you'd have to wait for patches to get backported to
> RHEL 8.
>
> You would also have to be cautious with security because the export
> name is supplied by the untrusted client.
>
> > For example:
> >
> > $ nbdkit tar file=vm.ova
> >
> > $ qemu-nbd --list
> > exports available: 2
> >  export: 'disk1.qcow2'
> >   size:  910848
> >   flags: 0x48f ( readonly flush fua df cache )
> >   min block: 512
> >   opt block: 4096
> >   max block: 33554432
> >   available meta contexts: 1
> >    base:allocation
> >  export: 'disk2.qcow2'
> >   size:  910848
> >   flags: 0x48f ( readonly flush fua df cache )
> >   min block: 512
> >   opt block: 4096
> >   max block: 33554432
> >   available meta contexts: 1
> >    base:allocation
> >
> > $  qemu-img convert -f qcow2 -O raw nbb://localhost/disk1.qcow2 disk1.raw
> >
> > $  qemu-img convert -f qcow2 -O raw nbb://localhost/disk2.qcow2 disk2.raw
>
> Yup, it'd be nice ...
>
> Also nbdkit doesn't support the extension for listing export names,
> and that's a bunch more work.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-p2v converts physical machines to virtual machines.  Boot with a
> live CD or over the network (PXE) and turn machines into KVM guests.
> http://libguestfs.org/virt-v2v
>