Re: [Qemu-devel] [RFC/PATCH] migration: SMRAM dirty bitmap not fetched from kvm-kmod and not send to destination

[Paolo Bonzini]

On 14/09/2016 09:55, Herongguang (Stephen) wrote:
> Hi,
> We found a problem that when a redhat 6 VM reboots (in grub countdown
> UI), migrating this VM will result in VM’s memory difference between
> source and destination side. The difference always resides in GPA
> 0xA0000~0xC0000, i.e. SMRAM area.
> 
> Occasionally this result in VM instruction emulation error in
> destination side.
> 
> After some digging, I think this is because in migration code, in
> migration_bitmap_sync(), only memory slots in address space
> address_space_memory’s dirty bitmap  fetched from kvm-kmod, while SMRAM
> memory slot, in address space smram_address_space’s dirty bitmap not
> fetched from kvm-kmod, thus modifications in SMRAM in source side are
> not sent to destination side.
> 
> I tried following patch, and this phenomenon does not happen anymore. Do
> you think this patch is OK or do you have better idea? Thanks.

Nice caatch!

I think the right solution here is to sync all RAM memory regions
instead of the address spaces.  You can do that by putting a notifier in
MemoryRegion; register the notifier in all the RAM creation functions
(basically after every mr->ram=true or mr->rom_device=true), and
unregister it in memory_region_destructor_ram.

Thanks,

Paolo

> diff --git a/migration/ram.c b/migration/ram.c
> index a3d70c4..1cc4360 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -607,6 +607,8 @@ static void migration_bitmap_sync_init(void)
>      iterations_prev = 0;
>  }
> 
> +extern AddressSpace smram_address_space;
> +
>  static void migration_bitmap_sync(void)
>  {
>      RAMBlock *block;
> @@ -627,6 +629,7 @@ static void migration_bitmap_sync(void)
> 
>      trace_migration_bitmap_sync_start();
>      address_space_sync_dirty_bitmap(&address_space_memory);
> +    address_space_sync_dirty_bitmap(&smram_address_space);
> 
>      qemu_mutex_lock(&migration_bitmap_mutex);
>      rcu_read_lock();
> diff --git a/target-i386/kvm.c b/target-i386/kvm.c
> index d1a25c5..b98fe22 100644
> --- a/target-i386/kvm.c
> +++ b/target-i386/kvm.c
> @@ -1111,7 +1111,7 @@ static int kvm_get_supported_msrs(KVMState *s)
> 
>  static Notifier smram_machine_done;
>  static KVMMemoryListener smram_listener;
> -static AddressSpace smram_address_space;
> +AddressSpace smram_address_space;
>  static MemoryRegion smram_as_root;
>  static MemoryRegion smram_as_mem;
> 
> 
>