Re: [Qemu-block] qemu-img: add seek option to dd

[Max Reitz]

On 23.02.19 11:13, Richard W.M. Jones wrote:
> Max:
> 
> What I actually omitted from this email is why I want to do this.
> I'm always complaining about other people doing that, sorry :-)
> 
> In nbdkit we have a thing called the "cow" [copy on write] filter.
> You can place it above other plugins and it adds a temporary writable
> overlay.  It's useful if the source plugin is read-only.  Or even if
> it is writable but you don't want to actually modify the source, while
> giving the impression to the NBD client that the disk is writable.
> Currently the cow filter is entirely temporary.  It is thrown away
> when nbdkit exits.
> 
> One thing that came out of discussions in FOSDEM is that it'd be a
> good idea to optionally allow the cow layer to be saved off on exit.
> It could be saved out as a qcow2 file.
> 
> Saying this is easy, but actually doing it is rather more complex.
> 
> The problem is that what is "under" the cow filter might not be a
> plain file, it could be all kinds of things.  For example:
> 
>   nbdkit --filter=cow --filter=xz file disk.xz
> 
> Under the cow filter is another filter which transparently
> xz-uncompresses the file.  There's no way to express this as a qcow2
> backing file.  (Even if there is something I've missed, there will
> definitely be cases where this is inexpressible in qemu/qcow2, for
> example a custom plugin or a non-free plugin.)
> 
> So from there I thought we could create a qcow2 file backed by
> ‘null-co://’.  The qcow2 file would contain only the blocks that are
> modified.  "Holes" in the qcow2 file would go through to the backing
> file and read as zeroes.

Maybe it'd be better to just put "garbage" into the backing file link so
qemu just cannot open the image at all?

(e.g. "xz://disk.xz" in the above case)

> Such a qcow2 file is not actually useful to be attached to a VM, but
> there are two things a user might do with it: (1) They could unsafe-
> rebase it (‘qemu-img rebase -u’) to change the backing file to a real
> file.  (2) We could provide a way to reload the qcow2 file back into
> the cow filter.
> 
> This led me to wonder how we could create such a qcow2 file using qemu
> tools.  Creating the empty qcow2 is easy enough.  However writing the
> blocks would require the qemu-img dd ‘seek’ operation.

I don't know how you're storing the overlay information, but if it's in
memory, I would wonder where you'd get the qemu-img dd input even from.
 Do you intend to write it to disk (e.g. tmpfs) first and then copy it
with qemu-img dd?  Especially in something that deals with nbd anyway
that seems like one step too much if you could just launch qemu-nbd on
the image and use NBD to write the data.

In addition to what Eric proposed, you can also use qemu-img convert in
a convoluted way to copy data to a given window of a target image, say
offset 1M and length 2M:

$ qemu-io -f qcow2 -c 'read -P 42 1M 2M' target.qcow2
Pattern verification failed at offset 1048576, 2097152 bytes
read 2097152/2097152 bytes at offset 1048576
2 MiB, 1 ops; 0.0005 sec (3.445 GiB/sec and 1763.6684 ops/sec)

$ qemu-img create -f raw source-window.raw 2M
$ qemu-io -f raw -c 'write -P 42 0 2M' source.raw

$ qemu-img convert -n -f raw --target-image-opts source.raw \
    driver=raw,offset=1M,size=2M,file.driver=qcow2,\
file.file.driver=file,file.file.filename=target.qcow2

$ qemu-io -f qcow2 -c 'read -P 42 1M 2M' target.qcow2
read 2097152/2097152 bytes at offset 1048576
2 MiB, 1 ops; 0.0018 sec (1.084 GiB/sec and 554.9390 ops/sec)


With qemu-blkfuse, you'd just do this:

$ qemu-blkfuse target.qcow2
$ dd if=source-window.raw of=target.qcow2 bs=1M seek=1 count=2 \
    conv=notrunc
$ fusermount -u target.qcow2

(Or you'd just write() to it yourself.)

The main issue with qemu-blkfuse is that it's slower than NBD, at least
right now.  It's still useful, I just need to finalize the patches...

> Anyway, you may well disagree with all or part of this analysis, and
> I'm really interested in what you think, negative or otherwise.

Mostly I'm wondering why something that deals with NBD does not use NBD
to write the data. :-)

Max

signature.asc
Description: OpenPGP digital signature