Re: [PATCH 0/3] qemu-img: Add convert --bitmaps

[Nir Soffer]

On Thu, Apr 16, 2020 at 5:51 PM Eric Blake <address@hidden> wrote:
>
> Without this series, the process for copying one qcow2 image to
> another including all of its bitmaps involves running qemu and doing
> the copying by hand with a series of QMP commands.  This makes the
> process a bit more convenient.

This seems good for copying an image chain from one storage to another,
but I think we need a similar --bitmaps option to qemu-img measure to make
this really useful.

Here is example use case showing how qemu-img measure is related:

Source chain:
/dev/vg1/base
/dev/vg1/top

Destination chain:
/dev/vg2/base
/dev/vg2/top

We create empty lvs with the same name on destination storage (/dev/vg2).

We measure the base lv using qemu-img measure for creating the target lv:

    qemu-img measure -f qcow2 -O qcow2 /dev/vg1/base
    lvcreate -L required_size /dev/vg2/base
    qemu-img create -f qcow2 /dev/vg2/base 10g

For the top lv we use the current size of the source lv - I think we
should measure it instead but
I'm not sure if qemu-img measure supports measuring a single image in a chain
(maybe -o backing_file?).

    lvcreate -L current_size /dev/vg2/top
    qemu-img create -f qcow2 -b /dev/vg2/base -F qcow2 /dev/vg2/top 10g

And then convert the lvs one by one:

    qemu-img convert -f qcow2 -O qcow2 -n --bitmaps /dev/vg1/base /dev/vg2/base
    qemu-img convert -f qcow2 -O qcow2 -n --bitmaps -B /dev/vg2/base
/dev/vg1/top /dev/vg2/top

The first copy may fail with ENOSPC since qemu-img measure of the base
does not consider the
bitmaps in the required size.

So I think we need to add a similar --bitmaps option to qemu-img
measure, hopefully reusing the
same code to find and estimate the size of the bitmaps.

Maybe we can estimate the size using qemu-img info --bitmaps, but I
think the right way to
do this is in qemu-img measure.

We have also another use case when we collapsed an image chain to single image:

Source chain:
/dev/vg1/base
/dev/vg1/top

Destination:
/dev/vg2/collapsed

In this case we measure the size of the entire chain (/dev/vg1/base <-
/dev/vg1/top) and create
/dev/vg2/collapsed in the correct size, and then we convert the chain using:

   qemu-img convert /dev/vg1/top /dev/vg2/collapsed

Currently we use this for exporting images, for example when creating
templates, or as a simple
backup. In this case we don't need to copy the bitmaps in the target
image - this is a new image
not used by any VM. Copying the bitmaps may also be non-trivial since
we may have the bitmaps
with the same names in several layers (e.g. result of live snapshot).

So I think using --bitmaps should be disabled when doing this kind of
convert. We can handle this
on our side easily, but I think this should fail or log a warning on
qemu-img, or require merging of
bitmaps with same names during the copy. I did not check if you
already handle this.

Finally we also have a use case when we copy the chain as is to new or
same storage, but
we create a new vm. In this case I don't think the backup history
makes sense for the new
vm, so we don't need to copy the bitmaps.

I will review the rest of the patches next week and can maybe give
this some testing.

Nir

> I still think that someday we will need a 'qemu-img bitmap' with
> various subcommands for manipulating bitmaps within an offline image,
> but in the meantime, this seems like a useful addition on its own.
>
> Series can also be downloaded at:
> https://repo.or.cz/qemu/ericb.git/shortlog/refs/tags/qemu-img-bitmaps-v1
>
> Eric Blake (3):
>   blockdev: Split off basic bitmap operations for qemu-img
>   qemu-img: Add convert --bitmaps option
>   iotests: Add test 291 to for qemu-img convert --bitmaps
>
>  docs/tools/qemu-img.rst    |   6 +-
>  Makefile.objs              |   2 +-
>  include/sysemu/blockdev.h  |  10 ++
>  blockbitmaps.c             | 217 +++++++++++++++++++++++++++++++++++++
>  blockdev.c                 | 184 -------------------------------
>  qemu-img.c                 |  81 +++++++++++++-
>  MAINTAINERS                |   1 +
>  qemu-img-cmds.hx           |   4 +-
>  tests/qemu-iotests/291     | 143 ++++++++++++++++++++++++
>  tests/qemu-iotests/291.out |  56 ++++++++++
>  tests/qemu-iotests/group   |   1 +
>  11 files changed, 514 insertions(+), 191 deletions(-)
>  create mode 100644 blockbitmaps.c
>  create mode 100755 tests/qemu-iotests/291
>  create mode 100644 tests/qemu-iotests/291.out
>
> --
> 2.26.0
>