Re: [Qemu-block] Converting qcow2 image on the fly to raw format

[Kevin Wolf]

Am 09.07.2018 um 18:52 hat Richard W.M. Jones geschrieben:
> On Mon, Jul 09, 2018 at 07:38:05PM +0300, Nir Soffer wrote:
> > We are discussing importing VM images to KubVirt. The goal is to be
> > able to import an existing qcow2 disk, probably some appliance stored
> > on http server, and and convert it to raw format for writing to storage.
> > 
> > This can be also useful for for oVirt for importing OVA, since we like to
> > pack
> > disks in qcow2 format inside OVA, but the user may like to use raw disks, or
> > for uploading existing disks.
> > 
> > Of course converting the image using qemu-img is easy, but requires
> > downloading the image to temporary disk. We would like to avoid temporary
> > disks, or telling users to convert the image.
> > 
> > Base on the discussion we had here:
> > https://lists.ovirt.org/archives/list/address@hidden/thread/GNAVJ253FP65QUSOONES5XZGRIDX5ABC/#YMLSEGU7PN3MX5MUORGEGGAQLLSL4KKJ
> > 
> > I think this is impossible since qcow2 is not built for streaming. But both
> > Richard and Eric suggested some solutions.
> > 
> > The flow is:
> > 
> >     qcow2 image -- http --> importer -> raw file
> > 
> > Is it possible to implement the importer using qemu-img and qemu-nbd,
> > or maybe nbdkit?
> 
> Strictly speaking streaming qcow2 to raw is not possible.  However
> placing an overlay on top of the original remote image will allow
> streaming to raw with only a modest amount of local storage consumed.
> 
> You can demonstrate this fairly easily:
> 
> $ qemu-img create -f qcow2 -b 'json: { "file.driver": "https", "file.url": 
> "https://uk-mirrors.evowise.com/fedora/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2";,
>  "file.timeout": 10000 }' /var/tmp/overlay.qcow2
> $ qemu-img convert -p -f qcow2 -O raw overlay.qcow2 fedora.img

This overlay stays empty, so it's pretty pointless and you could just
directly point 'qemu-img convert' to https and the real image.

> There are various tricks you can play with caching and copy-on-read to
> make the process more efficient (note that copy-on-read will result in
> a full local copy, but there are maybe some optimizations we could do
> in qcow2 for that).

Indeed, an overlay and copy-on-read can be useful if you want to start a
VM while the local copy isn't complete yet.

Ideally you'd just use the overlay as the target image then for best
efficiency, but obviously that requires that the user doesn't insist on
raw, which doesn't provide much feature-wise (including the backing
files that are required for this setup) and would complicate the
scenario considerably.

Kevin