[Qemu-devel] [RFC] migration next tree

[Juan Quintela]

Hi

As discussed on this week kvm call, here are the migration tree with
XBRLE + huge memory fixes integrated.  There were a bug on huge memory
(corrected on the v3 version just sent) and another on XZBRLE patches.
Orit is going to resend patches addressing comments, but here the fix is
integrated already.

How to test?

Compile qemu with trace support:

./configure --enable-trace-backend=simple

Create an event file:

$ cat /tmp/events
savevm_section_start
savevm_section_end
$

Now launch qemu with normal command line:

qemu ..... --trace events=/tmp/events

And when execution finish, you can use a script like the following to
see the stalls that happen:

$ cat ~/counter_simpletrace
$1 ~ /savevm_section_end/ {
        /* Print savevm_section_stop line when >100 ms duration */
        if ($2 > 50000) {
                printf("%s times_missing=%u\n", $0, times_missing++);
        }
}
$

And now you can analize:

./scripts/simpletrace.py ./trace-events trace-32554 | awk -f 
~/counter_simpletrace

(adjust paths as required)

Notice that this are stalls on the iothread.  vcpus stalls can still be
bigger.  The number that is tested here is 50ms, so I get cases that are
a bit bigger (between 60-100ms with one 8GB guest that is dirtying 512MB
in a loop.

If you get stalls, or drop connections, please let me know the load.

Thanks, Juan.


The following changes since commit 47ecbdf07ed2c37bdfd2d77137d01bb319ce13da:

  libcacard: build fixes (2012-06-21 20:04:24 +0000)

are available in the git repository at:

  http://repo.or.cz/r/qemu/quintela.git migration-next

for you to fetch changes up to 0844b46b5d7d8d6fa01134d170b3c48b7f32eab7:

  Add XBZRLE statistics (2012-06-22 15:24:15 +0200)

----------------------------------------------------------------
Isaku Yamahata (1):
      Add MigrationParams structure

Juan Quintela (8):
      Add spent time for migration
      Add tracepoints for savevm section start/end
      No need to iterate if we already are over the limit
      Only TCG needs TLB handling
      Only calculate expected_time for stage 2
      Exit loop if we have been there too long
      dirty bitmap: abstract its use
      Maintain the number of dirty pages

Orit Wasserman (13):
      Add missing check for host_from_stream_offset return value for 
RAM_SAVE_FLAG_PAGE
      Add migration capabilites
      Add XBZRLE documentation
      Add cache handling functions
      Add uleb encoding/decoding functions
      Add save_block_hdr function
      Add debugging infrastructure
      Change ram_save_block to return -1 if there are no more changes
      Add migration_end function
      Add xbzrle_encode_buffer and xbzrle_decode_buffer functions
      Add XBZRLE to ram_save_block and ram_save_live
      Add set_cachesize command
      Add XBZRLE statistics

 Makefile.objs        |    1 +
 arch_init.c          |  378 ++++++++++++++++++++++++++++++++++++++++++++------
 block-migration.c    |    8 +-
 cache.c              |  219 +++++++++++++++++++++++++++++
 cpu-all.h            |    1 +
 cutils.c             |   29 ++++
 docs/xbzrle.txt      |  115 +++++++++++++++
 exec-obsolete.h      |   50 ++++---
 exec.c               |   34 +++--
 hmp-commands.hx      |   34 +++++
 hmp.c                |   90 ++++++++++++
 hmp.h                |    3 +
 include/qemu/cache.h |   81 +++++++++++
 migration.c          |  131 +++++++++++++++--
 migration.h          |   30 +++-
 monitor.c            |    7 +
 qapi-schema.json     |   97 ++++++++++++-
 qemu-common.h        |   19 +++
 qmp-commands.hx      |   99 +++++++++++++
 savevm.c             |  115 ++++++++++++++-
 sysemu.h             |    3 +-
 trace-events         |    5 +
 vmstate.h            |    2 +-
 23 files changed, 1449 insertions(+), 102 deletions(-)
 create mode 100644 cache.c
 create mode 100644 docs/xbzrle.txt
 create mode 100644 include/qemu/cache.h