[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-discuss] vm live storage migration with snapshots

From: Edward Young
Subject: Re: [Qemu-discuss] vm live storage migration with snapshots
Date: Wed, 11 Feb 2015 15:08:25 -0600

Great! Thank you Jakob!


On Wed, Feb 11, 2015 at 2:08 PM, Jakob Bohm <address@hidden> wrote:
On 11/02/2015 20:42, Edward Young wrote:
Hello Jakob,

Thank you very much for quick and valuable answers to my question!

I have follow up questions about this topic:

1. Is there any other way we can finish this process without the shared
storage? I see the last tip you mentioned, we still use a tempsnap in
shared nfs server. If we consider the long distance VM migration
scenario, we do not have any shared storage resource between the source
and destination.

One way would be to make the top snapshot a qcow2 snapshot
created with a vmsave memory snapshot included, force off
the machine (virsh destroy), rsync any leftover changes to
the new locations, then resume the new machine from the
snapshot.  This means downtime.

Another way would be to set up some form of simple file
sharing (NFS, SMB, or similar) for just enough storage to
store the tempsnap small snapshot.  This would typically
be much smaller than the capacity needed to store a full
disk even for a single VM, provided you do the procedure
steps 2-5 (temporary snap, final rsync, migrate, commit)
on one VM at a time.  Assuming of course that nothing
inside the VM decides to do a full disk rewrite (defrag,
zerofree, read-write disk test etc.) in the middle of
the process, as that would explode the temporary
snapshot to full disk size.

2. I noticed in the qemu implementation, we have drive_mirror function.
The drive_mirror will mirror the write requests to both the source and
the destination. Can we use drive_mirror to transfer the new generated
data in the migration. In the background, we copy all the snapshots to
the destination. After the transformation is done, we reconstruct the
disk images in the destination. I'm not sure whether this is feasible or

I don't know that option, so cannot advise on its abilities.

    *From: *Jakob Bohm <address@hidden
    *Date: *February 11, 2015 11:38:15 AM
    *To: *address@hidden <mailto:address@hiddenorg>
    *Subject: **Re: [Qemu-discuss] vm live storage migration with

    On 11/02/2015 18:08, Edward Young wrote:
    Hi all,

    I'm investigating the ways to improve the live migration
    performance in libvirt. I have a question about the vm live
    storage migration. the platform is libvirt + qemu + kvm

    If we want to migrate a running vm with its virtual disk images
    to another node.
    we can use 'virsh migrate ....' commands.

    What if this vm has a number of disk-only external snapshots? In
    the current version, how can live migrate this vm?

    Is it possible to blockcommit the snapshots to the base image and
    later perform the migration, without shutting down the running vm?

    Or is it possible to iteratively transfer all the snapshots to
    the destination and later live migrate only the latest new data?

    Thanks in advance.

    Just a quick answer, others can answer on clever ways to
    transfer disk deltas between systems:

    - The usual way is to store both the base images and the
     deltas on some kind of shared storage accessible from
     both hosts involved in the migration, thus eliminating
     the need to transfer any disk contents.

    - To the extent that the (not changing at that point in
     time) base image and deltas for the non-current
     snapshot are stored in separate files, they could
     simply be transferred with ordinary tools such as cp, dd
     or rsync.

    - If you freeze execution of the guest during migration
     (not usually a preferred thing when wanting to do live
     migration), remaining deltas could be transferred with a
     simple rsync.

    - If you can find a way to make the snapshot commands
     store all further deltas around the time (just
     before/after) the migration in a separate delta file on
     a shared drive (NFS etc.), you could do that, then rsync
     all the (now stable) underlying files, then live migrate
     the running VM from using the shared delta and one hosts
     underlying files to using the same shared delta with the
     other hosts (identical) underlying files, then commit
     the snapshot delta from the NFS drive to the new host's
     copy of the next lower snapshot or base image.

    ASCII art illustration of the final tip above:

                |   VM   |
                 /      \
        +---------+    +---------+
        |  Host1  |    |  Host2  |
        +---------+    +---------+
          |      \      /      |
          | 2Snap \    /5Commit|
          |    +----------+    |
          |    | TempSnap |    |  On NFS etc.
          |    +----------+    |
          |      /      \      |
          |     / 3rsync \     |
        +---------+    +---------+
        |Snapshots|    |Snapshots|
        +---------+    +---------+
             |    1copy     |
        +---------+    +---------+
        |  Base   |    |  Base   |
        +---------+    +---------+

       Note that disks are only identical during step 4


Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

reply via email to

[Prev in Thread] Current Thread [Next in Thread]