Re: bitmaps -- copying allocation status into dirty bitmaps

[Eric Blake]

On 11/1/19 4:49 PM, Denis V. Lunev wrote:
> On 11/1/19 4:42 PM, John Snow wrote:
> > Hi, in one of my infamously unreadable and long status emails, I
> > mentioned possibly wanting to copy allocation data into bitmaps as a way
> > to enable users to create (external) snapshots from outside of the
> > libvirt/qemu context.
> >
> > (That is: to repair checkpoints in libvirt after a user extended the
> > backing chain themselves, you want to restore bitmap information for
> > that node. Conveniently, this information IS the allocation map, so we
> > can do this.)
> >
> > It came up at KVM Forum that we probably do want this, because oVirt
> > likes the idea of being able to manipulate these chains from outside of
> > libvirt/qemu.
> >
> > Denis suggested that instead of a new command, we can create a special
> > name -- maybe "#ALLOCATED" or something similar that can never be
> > allocated as a user-defined bitmap name -- as a special source for the
> > merge command.
> >
> > You'd issue a merge from "#ALLOCATED" to "myBitmap0" to copy the current
> > allocation data into "myBitmap0", for instance.
> original problem was a little bit incorrect. After some thoughts I found
> that this is NOT enough. We should also add zeroed clusters to the
> bitmap to merge! They do cover some data clusters from the original
> image.
>
> Thus we should either provide "ALLOCATED" bitmap for other purposes,
> and we should supply "CHANGED" which contains allocated AND
> explicitly zeroed clusters.

I'm also wondering if 'nbd-server-add' should add support to expose a 
'qemu:allocation:XXX' meta context, in addition to the existing 
'qemu:dirty-bitmap:XXX' contexts (would it just be a 0/1 bit for what is 
allocated in block layer XXX, or would it be an integer depth 0,1,2,... 
based on how deep in the chain a cluster is allocated, or?)

It may also be interesting to have a way to have 'nbd-server-add' force 
EIO errors to reads from an area not covered by a bitmap (whether that 
be the dirty bitmap or the #ALLOCATED bitmap), rather than falling back 
to a read from the backing chain, to ensure that an NBD client using 
such a backup image cannot read any more data than what the bitmap says 
is available.

> Den
> > Some thoughts:
> >
> > - The only commands where this pseudo-bitmap makes sense is merge.
> > enable/disable/remove/clear/add don't make sense here.
> >
> > - This pseudo bitmap might make sense for backup, but it's not needed;
> > you can just merge into an empty/enabled bitmap and then use that.
> >
> > - Creating an allocation bitmap on-the-fly is probably not possible
> > directly in the merge command, because the disk status calls might take
> > too long...
> >
> > Hm, actually, I'm not sure how to solve that one. Merge would need to
> > become a job (or an async QMP command?) or we'd need to keep an
> > allocation bitmap object around and in-sync. I don't really want to do
> > either, so maybe I'm missing an obvious/better solution.
> >
> > Also, with regards to introspection, if we do create a special reserved
> > name like #ALLOCATED, we need to make sure that this is available and
> > obvious via the QAPI schema.

If nothing else, the recent addition of introspectible QMP features 
should make this possible.


--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org