Re: [Qemu-block] qemu-img: add seek option to dd

[Eric Blake]

On 2/23/19 4:13 AM, Richard W.M. Jones wrote:

> 
> So from there I thought we could create a qcow2 file backed by
> ‘null-co://’.  The qcow2 file would contain only the blocks that are
> modified.  "Holes" in the qcow2 file would go through to the backing
> file and read as zeroes.
> 
> Such a qcow2 file is not actually useful to be attached to a VM, but
> there are two things a user might do with it: (1) They could unsafe-
> rebase it (‘qemu-img rebase -u’) to change the backing file to a real
> file.  (2) We could provide a way to reload the qcow2 file back into
> the cow filter.
> 
> This led me to wonder how we could create such a qcow2 file using qemu
> tools.  Creating the empty qcow2 is easy enough.  However writing the
> blocks would require the qemu-img dd ‘seek’ operation.

One trick I learned and used for my KVM Forum demonstration is that you
can use qcow2 backed by the source (whether via an NBD connection, or
directly backed by the COW file in /tmp) NBD, coupled with copy-on-read,
then match up targetted reads to just the portions of the disk you want
populated in the qcow2.  Something like this (you'd have to flesh it out
into working code, but hopefully it shows the ideas):

system("qemu-img create -f qcow2 -b /tmp/nbdcow.xxxxxx -F raw save.qcow2");
FILE *f = popen("qemu-io -C -f qcow2 save.qcow2");
foreach(segment of /tmp/nbdcow.xxxxxx) {
  fprintf(f, "read %lld %lld\n", segment.offset, segment.length);
}
fprintf(f, "quit");
pclose(f);
system("qemu-img rebase -u -f qcow2 -b null-co:// save.qcow2");

That solves the question of how to populate the save.qcow2 image as
nbdkit --filter=cow exits.  Your other question, of how to go in
reverse, in order to get from a saved qcow2 image into a prepopulated
COW cache (presuming, of course, that whatever the --filter=cow is put
on top of is unchanged from last time), can utilize "qemu-img map
--output=json -f qocw2 save.qcow2" to learn which portions of the qcow2
file are interesting (it gives you both the guest offset that contains
data, and the host offset to which portion of the .qcow2 file serves
that data), and then read() those into memory or dd them into a new
temporary file.

> Anyway, you may well disagree with all or part of this analysis, and
> I'm really interested in what you think, negative or otherwise.

Yes, being able to tell qemu-img convert to copy only a subset of
guest-visible contents from one source into a possibly different
guest-visible offset of a destination would be useful. qemu-img dd does
not yet have full feature parity, but I'm reluctant to duplicate efforts
in isolation that would be better done universally in qemu-img convert.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org