[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-block] qemu-img: add seek option to dd
From: |
Max Reitz |
Subject: |
Re: [Qemu-block] qemu-img: add seek option to dd |
Date: |
Mon, 25 Feb 2019 13:52:49 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 |
On 23.02.19 11:13, Richard W.M. Jones wrote:
> Max:
>
> What I actually omitted from this email is why I want to do this.
> I'm always complaining about other people doing that, sorry :-)
>
> In nbdkit we have a thing called the "cow" [copy on write] filter.
> You can place it above other plugins and it adds a temporary writable
> overlay. It's useful if the source plugin is read-only. Or even if
> it is writable but you don't want to actually modify the source, while
> giving the impression to the NBD client that the disk is writable.
> Currently the cow filter is entirely temporary. It is thrown away
> when nbdkit exits.
>
> One thing that came out of discussions in FOSDEM is that it'd be a
> good idea to optionally allow the cow layer to be saved off on exit.
> It could be saved out as a qcow2 file.
>
> Saying this is easy, but actually doing it is rather more complex.
>
> The problem is that what is "under" the cow filter might not be a
> plain file, it could be all kinds of things. For example:
>
> nbdkit --filter=cow --filter=xz file disk.xz
>
> Under the cow filter is another filter which transparently
> xz-uncompresses the file. There's no way to express this as a qcow2
> backing file. (Even if there is something I've missed, there will
> definitely be cases where this is inexpressible in qemu/qcow2, for
> example a custom plugin or a non-free plugin.)
>
> So from there I thought we could create a qcow2 file backed by
> ‘null-co://’. The qcow2 file would contain only the blocks that are
> modified. "Holes" in the qcow2 file would go through to the backing
> file and read as zeroes.
Maybe it'd be better to just put "garbage" into the backing file link so
qemu just cannot open the image at all?
(e.g. "xz://disk.xz" in the above case)
> Such a qcow2 file is not actually useful to be attached to a VM, but
> there are two things a user might do with it: (1) They could unsafe-
> rebase it (‘qemu-img rebase -u’) to change the backing file to a real
> file. (2) We could provide a way to reload the qcow2 file back into
> the cow filter.
>
> This led me to wonder how we could create such a qcow2 file using qemu
> tools. Creating the empty qcow2 is easy enough. However writing the
> blocks would require the qemu-img dd ‘seek’ operation.
I don't know how you're storing the overlay information, but if it's in
memory, I would wonder where you'd get the qemu-img dd input even from.
Do you intend to write it to disk (e.g. tmpfs) first and then copy it
with qemu-img dd? Especially in something that deals with nbd anyway
that seems like one step too much if you could just launch qemu-nbd on
the image and use NBD to write the data.
In addition to what Eric proposed, you can also use qemu-img convert in
a convoluted way to copy data to a given window of a target image, say
offset 1M and length 2M:
$ qemu-io -f qcow2 -c 'read -P 42 1M 2M' target.qcow2
Pattern verification failed at offset 1048576, 2097152 bytes
read 2097152/2097152 bytes at offset 1048576
2 MiB, 1 ops; 0.0005 sec (3.445 GiB/sec and 1763.6684 ops/sec)
$ qemu-img create -f raw source-window.raw 2M
$ qemu-io -f raw -c 'write -P 42 0 2M' source.raw
$ qemu-img convert -n -f raw --target-image-opts source.raw \
driver=raw,offset=1M,size=2M,file.driver=qcow2,\
file.file.driver=file,file.file.filename=target.qcow2
$ qemu-io -f qcow2 -c 'read -P 42 1M 2M' target.qcow2
read 2097152/2097152 bytes at offset 1048576
2 MiB, 1 ops; 0.0018 sec (1.084 GiB/sec and 554.9390 ops/sec)
With qemu-blkfuse, you'd just do this:
$ qemu-blkfuse target.qcow2
$ dd if=source-window.raw of=target.qcow2 bs=1M seek=1 count=2 \
conv=notrunc
$ fusermount -u target.qcow2
(Or you'd just write() to it yourself.)
The main issue with qemu-blkfuse is that it's slower than NBD, at least
right now. It's still useful, I just need to finalize the patches...
> Anyway, you may well disagree with all or part of this analysis, and
> I'm really interested in what you think, negative or otherwise.
Mostly I'm wondering why something that deals with NBD does not use NBD
to write the data. :-)
Max
signature.asc
Description: OpenPGP digital signature