[Qemu-devel] [RFC PATCH 0/8] job-based mirroring implementation

[Paolo Bonzini]

Hi all,

this is an alternative implementation of mirroring using a block job.
In the end this is the same as Marcelo's original live block copy code,
only done on top of the job infrastructure and coroutines.  Here, the
drive-mirror command sets up a job that copies data using the existing
dirty-bitmap functionality.  The target is treated as a "black box"
and data is copied from the source to the target in the background.

The mirror job is never-ending, but it is logically structured into
two phases: 1) copy all data as fast as possible until the target
first gets in sync with the source; 2) keep target in sync and
ensure that reopening to the target gets a correct (full) copy
of the source data.

The second phase is indicated by the progress in "info block-jobs"
reporting the current offset to be equal to the length of the file.
When the job is cancelled in the second phase, QEMU will run the
job until the source is clean and quiescent, then it will report
successful completion of the job.  (Note that it could already happen
that management lost the race against QEMU and got a completion
event instead of cancellation).

Compared to blkmirror, collapsing a chain of images to a new destination
just works, plus there are no strange interactions with other parts of
QEMU, no shared backing files.  Because the source is busy in a job, it
won't allow live snapshots and so on.

On the other hand the target will not be a perfect byte-by-byte copy
of the source, due to the granularity of the dirty bitmap.  The code
is more complex than blkmirror.  But then blkmirror's is really only
apparent simplicity.

Like Marcelo's code, these patches have the problem of communicating to
management the outcome of drive-reopen in a reliable manner.  For this,
patch 7 adds an option to drive-reopen to write a single byte to a file
passed via SCM_RIGHTS.  IIUC this was rejected in the past, but since
now blkmirror has also been rejected there's not much else to do...

I'm not really sure Kevin would welcome this functionality in 1.1; but
I've been asked to post it for comments, so I'm doing it before the soft
feature freeze.  It depends on the other patches in my queue, and in
particular on those moving generic functionality out of block/stream.c.

These patches do not make drive-mirror a transactionable command, but
this is just for simplicity.  It can be added on top.

Federico Simoncelli (1):
  block: add the drive-reopen command

Paolo Bonzini (7):
  block: introduce new dirty bitmap functionality
  block: allow interrupting a co_sleep_ns
  block: allow doing I/O in a job after cancellation
  block: add mirror job
  block: add drive-mirror command
  block: add witness argument to drive-reopen
  docs: add mirroring to live block operations

 Makefile.objs           |    2 +-
 block.c                 |   59 ++++++++++--
 block.h                 |    5 +-
 block/mirror.c          |  236 +++++++++++++++++++++++++++++++++++++++++++++++
 block/stream.c          |    7 +-
 block_int.h             |   17 +++-
 blockdev.c              |  195 ++++++++++++++++++++++++++++++++++++++-
 docs/live-block-ops.txt |   38 ++++++--
 hmp-commands.hx         |   37 ++++++++
 hmp.c                   |   37 ++++++++
 hmp.h                   |    2 +
 qapi-schema.json        |   60 ++++++++++++
 qemu-coroutine-sleep.c  |    3 +-
 qmp-commands.hx         |   76 +++++++++++++++
 trace-events            |    6 +-
 15 files changed, 751 insertions(+), 29 deletions(-)
 create mode 100644 block/mirror.c

-- 
1.7.9.3