[Qemu-devel] Re: QEMU interfaces for image streaming and post-copy block migration

[Daniel P. Berrange]

On Tue, Sep 07, 2010 at 08:41:44AM -0500, Anthony Liguori wrote:
> Hi,
> 
> We've got copy-on-read and image streaming working in QED and before 
> going much further, I wanted to bounce some interfaces off of the 
> libvirt folks to make sure our final interface makes sense.
> 
> Here's the basic idea:

[snip]

> A related topic is block migration.  Today we support pre-copy migration 
> which means we transfer the block device and then do a live migration.  
> Another approach is to do a live migration, and on the source, run a 
> block server using image streaming on the destination to move the device.
> 
> With QED, to implement this one would:
> 
> 1) launch qemu-nbd on the source while the guest is running
> 2) create a qed file on the destination with copy-on-read enabled and a 
> backing file using nbd: to point to the source qemu-nbd
> 3) run qemu -incoming on the destination with the qed file
> 4) execute the migration
> 5) when migration completes, begin streaming on the destination to 
> complete the copy
> 6) when the streaming is complete, shut down the qemu-nbd instance on 
> the source

IMHO, adding further network sockets is the one thing we absolutely
don't want to do to migration. I don't much like the idea of launching
extra daemons either.

> This is a bit involved and we could potentially automate some of this in 
> qemu by launching qemu-nbd and providing commands to do some of this.  
> Again though, I think the question is what type of interfaces would 
> libvirt prefer?  Low level interfaces + recipes on how to do high level 
> things or higher level interfaces?

I think it should be done entirely within the main QEMU migration
socket. I know this isn't possible with the current impl, since it
is unidirectional, preventing the target sending the source requests
for specific data blocks. If we made migration socket bi-directional
I think we could do it all within qemu with no external helpers
or extra sockets

 1. Create empty qed file on the destination with copy on read
    enable backing file pointing to a special 'migrate:' protocol
 2. Run qemu -incoming on the destination with with the qed file
 3. execute the migration
 4. when migration completes, target QEMU continues streaming blocks
    from the soruce qemu.
 5. when streaming is complete, source qemu can shutdown.


Both your original proposal and mine here seem to have a pretty
bad failure scenario though. After the cut-over point where the
VM cpus start running on the destination QEMU, AFAICT, any failure
on the source before block streaming complete leaves you dead in
the water.  The source VM no longer has up2date RAM contents and
the destination VM does not yet have a complete disk image.

Regards,
Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|