qemu-discuss
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-discuss] Converting qcow2 image on the fly to raw format


From: Adam Litke
Subject: Re: [Qemu-discuss] Converting qcow2 image on the fly to raw format
Date: Wed, 11 Jul 2018 14:17:18 +0300

Adding some kubevirt developers to the thread.  Thanks guys for the
information!  I think this could work perfectly for on the fly conversion
of qcow2 images to raw format on our PVCs.

On Mon, Jul 9, 2018 at 9:47 PM Nir Soffer <address@hidden> wrote:

> On Mon, Jul 9, 2018 at 8:06 PM Richard W.M. Jones <address@hidden>
> wrote:
>
>> On Mon, Jul 09, 2018 at 07:02:50PM +0200, Kevin Wolf wrote:
>> > Am 09.07.2018 um 18:52 hat Richard W.M. Jones geschrieben:
>> > > On Mon, Jul 09, 2018 at 07:38:05PM +0300, Nir Soffer wrote:
>> > > > We are discussing importing VM images to KubVirt. The goal is to be
>> > > > able to import an existing qcow2 disk, probably some appliance
>> stored
>> > > > on http server, and and convert it to raw format for writing to
>> storage.
>> > > >
>> > > > This can be also useful for for oVirt for importing OVA, since we
>> like to
>> > > > pack
>> > > > disks in qcow2 format inside OVA, but the user may like to use raw
>> disks, or
>> > > > for uploading existing disks.
>> > > >
>> > > > Of course converting the image using qemu-img is easy, but requires
>> > > > downloading the image to temporary disk. We would like to avoid
>> temporary
>> > > > disks, or telling users to convert the image.
>> > > >
>> > > > Base on the discussion we had here:
>> > > >
>> https://lists.ovirt.org/archives/list/address@hidden/thread/GNAVJ253FP65QUSOONES5XZGRIDX5ABC/#YMLSEGU7PN3MX5MUORGEGGAQLLSL4KKJ
>> > > >
>> > > > I think this is impossible since qcow2 is not built for streaming.
>> But both
>> > > > Richard and Eric suggested some solutions.
>> > > >
>> > > > The flow is:
>> > > >
>> > > >     qcow2 image -- http --> importer -> raw file
>> > > >
>> > > > Is it possible to implement the importer using qemu-img and
>> qemu-nbd,
>> > > > or maybe nbdkit?
>> > >
>> > > Strictly speaking streaming qcow2 to raw is not possible.  However
>> > > placing an overlay on top of the original remote image will allow
>> > > streaming to raw with only a modest amount of local storage consumed.
>> > >
>> > > You can demonstrate this fairly easily:
>> > >
>> > > $ qemu-img create -f qcow2 -b 'json: { "file.driver": "https",
>> "file.url": "
>> https://uk-mirrors.evowise.com/fedora/releases/28/Cloud/x86_64/images/Fedora-Cloud-Base-28-1.1.x86_64.qcow2";,
>> "file.timeout": 10000 }' /var/tmp/overlay.qcow2
>> > > $ qemu-img convert -p -f qcow2 -O raw overlay.qcow2 fedora.img
>> >
>> > This overlay stays empty, so it's pretty pointless and you could just
>> > directly point 'qemu-img convert' to https and the real image.
>>
>> Right, indeed.  I was copying what virt-v2v does without thinking
>> about it enough.  virt-v2v needs the overlay because it actually wants
>> to write into it, and it does copy-on-read for the first phase (not
>> the final ‘qemu-img convert’).
>>
>
> Thanks, I just tested the simple:
>
>     qemu-img convert -p -f qcow2 -O raw http://localhost/orig.qcow2
> converted.raw
>
> And it just works :-)
>
> I got timeouts trying to download from
>
> https://download.fedoraproject.org/pub/alt/atomic/stable/Fedora-Atomic-28-20180625.1/AtomicHost/x86_64/images/Fedora-AtomicHost-28-20180625.1.x86_64.qcow2
>
> I guess we need to use
> 'json: { "file.driver": "http", "file.url": "url...", "file.timeout":
> 10000 }'
> To change timeout? Where is these and other options documented?
>
> I did also some timings, using sever on local network with 1g nic.
>
> $ time wget
> http://local.server/Fedora-AtomicHost-28-20180625.1.x86_64.qcow2
> ...
> Length: 638043136 (608M) [application/octet-stream]Saving to:
> ‘Fedora-AtomicHost-28-20180625.1.x86_64.qcow2’
>
> Fedora-AtomicHost-28-20180625.1.x86_64.qcow2
> 100%[=====================================================================================================>]
> 608.49M   107MB/s    in 5.6s
>
> 2018-07-09 21:38:39 (108 MB/s) -
> ‘Fedora-AtomicHost-28-20180625.1.x86_64.qcow2’ saved [638043136/638043136]
>
>
> real 0m5.941s
> user 0m0.183s
> sys 0m1.185s
>
> $ time qemu-img convert -p -f qcow2 -O raw
> http://local.server/Fedora-AtomicHost-28-20180625.1.x86_64.qcow2
> converted.raw
>     (100.00/100%)
>
> real 0m14.217s
> user 0m5.235s
> sys 0m2.343s
>
> $ time qemu-img convert -p -f qcow2 -O raw
> Fedora-AtomicHost-28-20180625.1.x86_64.qcow2 converted.raw
>     (100.00/100%)
>
> real 0m11.909s
> user 0m4.728s
> sys 0m1.595s
>
> So converting on the fly is even little faster then downloading to
> temporary file and converting.
>
> Nir
>


-- 
Adam Litke


reply via email to

[Prev in Thread] Current Thread [Next in Thread]