Re: [Qemu-block] [ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Tomáš Golembiovský]

On Mon, 28 May 2018 13:37:59 +0200
Kevin Wolf <address@hidden> wrote:

> Am 28.05.2018 um 12:27 hat Arik Hadas geschrieben:
> > On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf <address@hidden> wrote:
> >   
> > > [ Adding qemu-block ]
> > >
> > > Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:  
> > > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer <address@hidden> wrote:
> > > >  
> > > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <  
> > > address@hidden>  
> > > > > wrote:
> > > > >  
> > > > >> Dear Nir,
> > > > >>
> > > > >> Thank you for quick reply.
> > > > >>
> > > > >> Ok, why it will not work?
> > > > >>  
> > > > >
> > > > > Because the image has a backing file which is not accessible to oVirt.
> > > > >
> > > > >  
> > > > >> I used qemu+tcp connection, via import method through engine admin 
> > > > >> UI.
> > > > >>
> > > > >> Images was imported and converted according logs, still “backing 
> > > > >> file”
> > > > >> invalid entry remained.
> > > > >>
> > > > >> Also, I did use same method before, connecting to plain “libvirt kvm”
> > > > >> host, import and conversion went smooth, no backend file.
> > > > >>
> > > > >> Image format is qcow(2) which is supported by oVirt.
> > > > >>
> > > > >> What am I missing? Should I use different method?
> > > > >>  
> > > > >
> > > > > I guess this is not a problem on your side, but a bug in our side.
> > > > >
> > > > > Either we should block the operation that cannot work, or fix the  
> > > process  
> > > > > so we don't refer to non-existing image.
> > > > >
> > > > > When importing we have 2 options:
> > > > >
> > > > > - import the entire chain,  importing all images in the chain,  
> > > converting  
> > > > >  each image to oVirt volume, and updating the backing file of each  
> > > layer  
> > > > > to point to the oVirt image.
> > > > >
> > > > > - import the current state of the image into a new image, using 
> > > > > either  
> > > raw  
> > > > > or qcow2, but without any backing file.
> > > > >
> > > > > Arik, do you know why we create qcow2 file with invalid backing file?
> > > > >  
> > > >
> > > > It seems to be a result of a bit naive behavior of the kvm2ovirt module
> > > > that tries to download only the top-level volume the VM uses, assuming  
> > > each  
> > > > of the disks to be imported is comprised of a single volume.
> > > >
> > > > Maybe it's time to finally asking QEMU guys to provide a way to consume 
> > > >  
> > > the  
> > > > 'collapsed' form of a chain of volumes as a stream if that's not  
> > > available  
> > > > yet? ;) It can also boost the recently added process of exporting VMs as
> > > > OVAs...  
> > >
> > > Not sure which operation we're talking about on the QEMU level, but
> > > generally the "collapsed" view is the normal thing because that's what
> > > guests see.
> > >
> > > For example, if you use 'qemu-img convert', you have to pass options to
> > > specifically disable it and convert only a single layer if you want to
> > > keep using backing files instead of getting a standalone image that
> > > contains everything.
> > >  
> > 
> > Yeah, some context was missing. Sorry about that.
> > 
> > Let me demonstrate briefly the flow for OVA:
> > Let's say that we have a VM that is based on a template and has one disk
> > and one snapshot, so its volume-chain would be:
> > T -> S -> V
> > (V is the volume the VM writes to, S is the backing file of V and T is the
> > backing file of S).
> > When exporting that VM to an OVA file we want the produced tar file to be
> > comprised of:
> > (1) OVF configuration
> > (2) single disk volume (preferably qcow).
> > 
> > So we need to collapse T, S, V into a single volume.
> > Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> > (a) qemu-img convert produces a 'temporary' collapsed volume
> > (b) make a tar file of the OVf configuration and that 'temporary' volume
> > (c) delete the temporary volume
> > 
> > But the fact that we produce that 'temporary' volume obviously slows down
> > the entire operation.
> > It would be much better if we could "open" a stream that we can read from
> > the 'collapsed' form of that chain and stream it directly into the
> > appropriate tar file entry, without extra writes to the storage device.
> > 
> > Few months ago people from the oVirt-storage team checked the qemu toolset
> > and replied that this capability is not yet provided, therefore we
> > implemented the workaround described above. Apparently, the desired ability
> > can also be useful for the flow discussed in this thread so it worth asking
> > for it again :)  
> 
> I think real streaming is unlikely to happen because most image formats
> that QEMU supports aren't made that way. If there is a compelling
> reason, we can consider it, but it would work only with very few target
> formats and as such would have to be separate from existing commands.
> 
> As for OVA files, I think it might be useful to have a tar block driver
> instead which would allow you to open a file inside a tar archive (you
> could then also directly run an OVA without extracting it first). We
> probably wouldn't be able to support resizing images there, but that
> should be okay.

That's something you can do with offset/size options too. In fact, that
was main reason for adding it so that virt-v2v can convert OVAs without
extracting them.

Having a layer that understands tar format and would supply offset/size
for you would be neat.

> If you can create a tar file that reserves space for the image file
> without actually writing it, a possible workaround today would be using
> the offset/size runtime options of the raw driver to convert directly
> into a region inside the tar archive.

Not easy to do for qcow2 where you don't know how much space you will
actually need.

    Tomas

> 
> Kevin
> _______________________________________________
> Users mailing list -- address@hidden
> To unsubscribe send an email to address@hidden
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: 
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives: 
> https://lists.ovirt.org/archives/list/address@hidden/message/EWEKAIIDE3SYAZVTGBVPSUJJQGNXQAVN/


-- 
Tomáš Golembiovský <address@hidden>