Re: [Qemu-block] [ovirt-users] Re: Libvirt ERROR cannot access backing file after importing VM from OpenStack

[Kevin Wolf]

Am 28.05.2018 um 16:06 hat Tomáš Golembiovský geschrieben:
> 
> On Mon, 28 May 2018 13:37:59 +0200
> Kevin Wolf <address@hidden> wrote:
> 
> > Am 28.05.2018 um 12:27 hat Arik Hadas geschrieben:
> > > On Mon, May 28, 2018 at 11:25 AM, Kevin Wolf <address@hidden> wrote:
> > >   
> > > > [ Adding qemu-block ]
> > > >
> > > > Am 27.05.2018 um 10:36 hat Arik Hadas geschrieben:  
> > > > > On Thu, May 24, 2018 at 6:13 PM, Nir Soffer <address@hidden> wrote:
> > > > >  
> > > > > > On Thu, May 24, 2018 at 6:06 PM Vrgotic, Marko <  
> > > > address@hidden>  
> > > > > > wrote:
> > > > > >  
> > > > > >> Dear Nir,
> > > > > >>
> > > > > >> Thank you for quick reply.
> > > > > >>
> > > > > >> Ok, why it will not work?
> > > > > >>  
> > > > > >
> > > > > > Because the image has a backing file which is not accessible to 
> > > > > > oVirt.
> > > > > >
> > > > > >  
> > > > > >> I used qemu+tcp connection, via import method through engine admin 
> > > > > >> UI.
> > > > > >>
> > > > > >> Images was imported and converted according logs, still “backing 
> > > > > >> file”
> > > > > >> invalid entry remained.
> > > > > >>
> > > > > >> Also, I did use same method before, connecting to plain “libvirt 
> > > > > >> kvm”
> > > > > >> host, import and conversion went smooth, no backend file.
> > > > > >>
> > > > > >> Image format is qcow(2) which is supported by oVirt.
> > > > > >>
> > > > > >> What am I missing? Should I use different method?
> > > > > >>  
> > > > > >
> > > > > > I guess this is not a problem on your side, but a bug in our side.
> > > > > >
> > > > > > Either we should block the operation that cannot work, or fix the  
> > > > process  
> > > > > > so we don't refer to non-existing image.
> > > > > >
> > > > > > When importing we have 2 options:
> > > > > >
> > > > > > - import the entire chain,  importing all images in the chain,  
> > > > converting  
> > > > > >  each image to oVirt volume, and updating the backing file of each  
> > > > layer  
> > > > > > to point to the oVirt image.
> > > > > >
> > > > > > - import the current state of the image into a new image, using 
> > > > > > either  
> > > > raw  
> > > > > > or qcow2, but without any backing file.
> > > > > >
> > > > > > Arik, do you know why we create qcow2 file with invalid backing 
> > > > > > file?
> > > > > >  
> > > > >
> > > > > It seems to be a result of a bit naive behavior of the kvm2ovirt 
> > > > > module
> > > > > that tries to download only the top-level volume the VM uses, 
> > > > > assuming  
> > > > each  
> > > > > of the disks to be imported is comprised of a single volume.
> > > > >
> > > > > Maybe it's time to finally asking QEMU guys to provide a way to 
> > > > > consume  
> > > > the  
> > > > > 'collapsed' form of a chain of volumes as a stream if that's not  
> > > > available  
> > > > > yet? ;) It can also boost the recently added process of exporting VMs 
> > > > > as
> > > > > OVAs...  
> > > >
> > > > Not sure which operation we're talking about on the QEMU level, but
> > > > generally the "collapsed" view is the normal thing because that's what
> > > > guests see.
> > > >
> > > > For example, if you use 'qemu-img convert', you have to pass options to
> > > > specifically disable it and convert only a single layer if you want to
> > > > keep using backing files instead of getting a standalone image that
> > > > contains everything.
> > > >  
> > > 
> > > Yeah, some context was missing. Sorry about that.
> > > 
> > > Let me demonstrate briefly the flow for OVA:
> > > Let's say that we have a VM that is based on a template and has one disk
> > > and one snapshot, so its volume-chain would be:
> > > T -> S -> V
> > > (V is the volume the VM writes to, S is the backing file of V and T is the
> > > backing file of S).
> > > When exporting that VM to an OVA file we want the produced tar file to be
> > > comprised of:
> > > (1) OVF configuration
> > > (2) single disk volume (preferably qcow).
> > > 
> > > So we need to collapse T, S, V into a single volume.
> > > Sure, we can do 'qemu-img convert'. That's what we do now in oVirt 4.2:
> > > (a) qemu-img convert produces a 'temporary' collapsed volume
> > > (b) make a tar file of the OVf configuration and that 'temporary' volume
> > > (c) delete the temporary volume
> > > 
> > > But the fact that we produce that 'temporary' volume obviously slows down
> > > the entire operation.
> > > It would be much better if we could "open" a stream that we can read from
> > > the 'collapsed' form of that chain and stream it directly into the
> > > appropriate tar file entry, without extra writes to the storage device.
> > > 
> > > Few months ago people from the oVirt-storage team checked the qemu toolset
> > > and replied that this capability is not yet provided, therefore we
> > > implemented the workaround described above. Apparently, the desired 
> > > ability
> > > can also be useful for the flow discussed in this thread so it worth 
> > > asking
> > > for it again :)  
> > 
> > I think real streaming is unlikely to happen because most image formats
> > that QEMU supports aren't made that way. If there is a compelling
> > reason, we can consider it, but it would work only with very few target
> > formats and as such would have to be separate from existing commands.
> > 
> > As for OVA files, I think it might be useful to have a tar block driver
> > instead which would allow you to open a file inside a tar archive (you
> > could then also directly run an OVA without extracting it first). We
> > probably wouldn't be able to support resizing images there, but that
> > should be okay.
> 
> That's something you can do with offset/size options too. In fact, that
> was main reason for adding it so that virt-v2v can convert OVAs without
> extracting them.
> 
> Having a layer that understands tar format and would supply offset/size
> for you would be neat.
> 
> > If you can create a tar file that reserves space for the image file
> > without actually writing it, a possible workaround today would be using
> > the offset/size runtime options of the raw driver to convert directly
> > into a region inside the tar archive.
> 
> Not easy to do for qcow2 where you don't know how much space you will
> actually need.

Shouldn't 'qemu-img measure' solve that problem these days?

Kevin