From: "Michael R. Hines" <address@hidden>
Changes since v8:
For very large virtual machines, pinning can take a long time.
While this does not affect the migration's *actual* time itself,
it is still important for the user to know what's going on and
to know what component of the total time is actual taken up by
pinning.
For example, using a 14GB virtual machine, pinning can take as
long as 5 seconds, for which the user would not otherwise know
what was happening.
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Paolo Bonzini <address@hidden>
Wiki: http://wiki.qemu.org/Features/RDMALiveMigration
Github: address@hidden:hinesmr/qemu.git
Here is a brief summary of total migration time and downtime using RDMA:
Using a 40gbps infiniband link performing a worst-case stress test,
using an 8GB RAM virtual machine:
Using the following command:
$ apt-get install stress
$ stress --vm-bytes 7500M --vm 1 --vm-keep
RESULTS:
1. Migration throughput: 26 gigabits/second.
2. Downtime (stop time) varies between 15 and 100 milliseconds.
EFFECTS of memory registration on bulk phase round:
For example, in the same 8GB RAM example with all 8GB of memory in
active use and the VM itself is completely idle using the same 40 gbps
infiniband link:
1. x-rdma-pin-all disabled total time: approximately 7.5 seconds @ 9.5 Gbps
2. x-rdma-pin-all enabled total time: approximately 4 seconds @ 26 Gbps
These numbers would of course scale up to whatever size virtual machine
you have to migrate using RDMA.
Enabling this feature does *not* have any measurable affect on
migration *downtime*. This is because, without this feature, all of the
memory will have already been registered already in advance during
the bulk round and does not need to be re-registered during the successive
iteration rounds.
The following changes since commit f3aa844bbb2922a5b8393d17620eca7d7e921ab3:
build: include config-{, all-}devices.mak after defining CONFIG_SOFTMMU and
CONFIG_USER_ONLY (2013-04-24 12:18:41 -0500)
are available in the git repository at:
address@hidden:hinesmr/qemu.git rdma_patch_v9
for you to fetch changes up to 75e6fac1f642885b93cefe6e1874d648e9850f8f:
rdma: send pc.ram (2013-04-24 14:55:01 -0400)
----------------------------------------------------------------
Michael R. Hines (14):
rdma: add documentation
rdma: introduce qemu_update_position()
rdma: export yield_until_fd_readable()
rdma: export throughput w/ MigrationStats QMP
rdma: introduce qemu_file_mode_is_not_valid()
rdma: export qemu_fflush()
rdma: introduce ram_handle_compressed()
rdma: introduce qemu_ram_foreach_block()
rdma: new QEMUFileOps hooks
rdma: introduce capability x-rdma-pin-all
rdma: core logic
rdma: send pc.ram
rdma: fix mlock() freezes and accounting
rdma: add pin-all accounting timestamp to QMP statistics
Makefile.objs | 1 +
arch_init.c | 69 +-
configure | 29 +
docs/rdma.txt | 415 ++++++
exec.c | 9 +
hmp.c | 6 +
include/block/coroutine.h | 6 +
include/exec/cpu-common.h | 5 +
include/migration/migration.h | 32 +
include/migration/qemu-file.h | 32 +
migration-rdma.c | 2831 +++++++++++++++++++++++++++++++++++++++++
migration.c | 36 +-
qapi-schema.json | 15 +-
qemu-coroutine-io.c | 23 +
savevm.c | 114 +-
15 files changed, 3574 insertions(+), 49 deletions(-)
create mode 100644 docs/rdma.txt
create mode 100644 migration-rdma.c