Re: [Qemu-devel] [PATCH v4 0/6] Efficient VM backup for qemu

[Stefan Hajnoczi]

On Thu, Feb 21, 2013 at 03:56:15PM +0000, Dietmar Maurer wrote:
> > On Wed, Feb 20, 2013 at 04:04:49PM +0000, Dietmar Maurer wrote:
> > > > The backup writer abstraction is a special case interface somewhere
> > > > between an image format and live migration.  I'd prefer it if the
> > > > backup block job stuck to BlockDriverState as the destination for backup
> > data.
> > > > Then you could save a point-in-time copy of a disk to a raw or even
> > > > qcow2 image.
> > >
> > > backup needs to work on non-seekable pipes.
> > 
> > In _your_ use case. That means that we should support using non-seekable
> > pipes, but it doesn't mean that any other use case is irrelevant. In fact I 
> > think
> > backing up to a normal raw or qcow2 image file is the interesting case for 
> > the
> > majority of people.
> 
> VMs can have more than one disk. How do you backup them into a single raw or 
> qcow2 image?
> And where do you store the configuration inside a qcow2 file?

Here is a draft showing how your backup block job can be used for your
VMA use case, as well as other use cases:

The QMP 'transaction' command takes a list of "actions" and executes
them atomically.  That means all actions either complete successfully or
they are all rolled back/cancelled.  Today it only supports taking
external snapshots of block devices, so you can say something like:

  transaction actions=[{'type': 'blockdev-snapshot-sync',
                        'device': 'virtio-drive0',
                        'snapshot-file': 'vm-snap1.qcow2'},
                       {'type': 'blockdev-snapshot-sync',
                        'device': 'virtio-drive1',
                        'snapshot-file': 'data-snap1.qcow2'}]

Now let's add your backup block job to QMP 'transaction'.  It will allow
the user to start your point-in-time copy block job on one or more block
devices atomically:

  transaction actions=[{'type': 'point-in-time-copy-async',
                        'device': 'virtio-drive0',
                        'destination-device': 'vm-snap1.qcow2'}]

This kicks off the backup block job.  The point of using transaction is
so we can start multiple block jobs atomically, guaranteeing a
consistent state across multiple block devices.

Now let's add live migrate to file (RAM + device state) to QMP
'transaction'.  This provides the functionality of your 'backup' command
minus the management tool's guest configuration file:

  transaction actions=[{'type': 'migrate-async',
                        'destination': 'file:vmstate.bin',
                        'id': 'migration'},
                       {'type': 'point-in-time-copy-async',
                        'wait-for': 'migration',
                        'device': 'virtio-drive0',
                        'destination-device': 'vm-snap1.qcow2'}]

I added the 'wait-for' attribute which tells QMP transaction to let the
migration action complete first before doing the backup block job.  This
way we first live migration VM state to vmstate.bin and then atomically
kick off the backup block job - with consistent disk state.

Finally, here is how VMA backup works.  There is an external vma-writer
program that works like this:

  $ vma-writer --output=backup-20130201.vma \
               --add-file=path/to/vm-config.xml \
               --vmstate=tcp:127.0.0.1:1234 \
               --blockdev=virtio-drive0,tcp:127.0.0.1:1235

It creates backup-20130201.vma and appends an arbitrary file with the VM
configuration.  It listens for vmstate (guest RAM + device state) on
TCP port 1234 and listens for virtio-drive0's backup over NBD on port 1235.

vma-writer starts by streaming the vmstate into the .vma file.  Next it
services zero or more NBD connections and puts all the WRITE data into
the .vma file.

Once the last NBD connection closes, vma-writer terminates and backup is
complete.

The QEMU command to initiate this backup looks something like:

  transaction actions=[{'type': 'migrate-async',
                        'destination': 'tcp:127.0.0.1:1234',
                        'id': 'migration'},
                       {'type': 'point-in-time-copy-async',
                        'wait-for': 'migration',
                        'device': 'virtio-drive0',
                        'destination-device': 'nbd:127.0.0.1:1235'}]

It's also possible to serve other use cases.  For example, I previously
showed how to take point-in-time copies of the disk without saving
vmstate.  And I also showed how to save vmstate to a file and each block
device to its own qcow2 file (i.e. the backup into a directory approach
that Paolo mentioned).

This approach is much more flexible than the monolithic 'backup'
command.  This is why I think it's the right approach to take - it can
implement use cases we haven't even encountered yet.

I also think this approach brings together your live backup work with
Wenchao's snapshot work.  It's how I imagined his backup use cases to
work at the QMP level:
http://wiki.qemu.org/Features/VMSnapshotEnchancement

Stefan