Re: [PATCH v5 0/8] virtio-mem: Handle preallocation with migration

[Michael S. Tsirkin]

On Tue, Jan 17, 2023 at 12:22:41PM +0100, David Hildenbrand wrote:
> While playing with migration of virtio-mem with an ordinary file backing,
> I realized that migration and prealloc doesn't currently work as expected
> for virtio-mem. Further, Jing Qi reported that setup issues (insufficient
> huge pages on the destination) result in QEMU getting killed with SIGBUS
> instead of failing gracefully.
> 
> In contrast to ordinary memory backend preallocation, virtio-mem
> preallocates memory before plugging blocks to the guest. Consequently,
> when migrating we are not actually preallocating on the destination but
> "only" migrate pages. Fix that be migrating the bitmap early, before any
> RAM content, and use that information to preallocate memory early, before
> migrating any RAM.
> 
> Postcopy needs some extra care, and I realized that prealloc+postcopy is
> shaky in general. Let's at least try to mimic what ordinary
> prealloc+postcopy does: temporarily allocate the memory, discard it, and
> cross fingers that we'll still have sufficient memory when postcopy
> actually tries placing pages.
> 
> Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
> Cc: Juan Quintela <quintela@redhat.com>
> Cc: Peter Xu <peterx@redhat.com>
> Cc: Michael S. Tsirkin <mst@redhat.com>
> Cc: Michal Privoznik <mprivozn@redhat.com>


Reviewed-by: Michael S. Tsirkin <mst@redhat.com>

> v4 -> v5:
> - "migration/savevm: Move more savevm handling into vmstate_save()"
> -- Extended patch description regarding tracing
> - "migration/savevm: Prepare vmdesc json writer in
>    qemu_savevm_state_setup()"
> -- Move freeing to migrate_fd_cleanup()
> - "migration/savevm: Allow immutable device state to be migrated early (i.e.,
>     before RAM)"
> -- "immutable" -> "early_setup"
> -- Extend comment
> - Added some RBs (thanks!)
> 
> v3 -> v4:
> - First 3 patches:
> -- Minimze code changes and simplify
> -- Save immutable device state during qemu_savevm_state_setup()
> -- Don't use vmsd priorities, use a new flag
> -- Split it logically up
> - "migration/ram: Factor out check for advised postcopy"
> -- Don't factor out postcopy_is_running()
> - "virtio-mem: Migrate immutable properties early"
> -- Adjust to changed vmsd interface
> - "virtio-mem: Proper support for preallocation with migration"
> -- Drop sanity check in virtio_mem_post_load_early()
> 
> v2 -> v3:
> - New approach/rewrite, drop RB and TB of last patch
> 
> v1 -> v2:
> - Added RBs and Tested-bys
> - "virtio-mem: Fail if a memory backend with "prealloc=on" is specified"
> -- Fail instead of warn
> -- Adjust subject/description
> 
> 
> David Hildenbrand (8):
>   migration/savevm: Move more savevm handling into vmstate_save()
>   migration/savevm: Prepare vmdesc json writer in
>     qemu_savevm_state_setup()
>   migration/savevm: Allow immutable device state to be migrated early
>     (i.e., before RAM)
>   migration/vmstate: Introduce VMSTATE_WITH_TMP_TEST() and
>     VMSTATE_BITMAP_TEST()
>   migration/ram: Factor out check for advised postcopy
>   virtio-mem: Fail if a memory backend with "prealloc=on" is specified
>   virtio-mem: Migrate immutable properties early
>   virtio-mem: Proper support for preallocation with migration
> 
>  hw/core/machine.c              |   4 +-
>  hw/virtio/virtio-mem.c         | 144 ++++++++++++++++++++++++++++++++-
>  include/hw/virtio/virtio-mem.h |   8 ++
>  include/migration/misc.h       |   4 +-
>  include/migration/vmstate.h    |  28 ++++++-
>  migration/migration.c          |   9 +++
>  migration/migration.h          |   4 +
>  migration/ram.c                |   8 +-
>  migration/savevm.c             | 105 +++++++++++++-----------
>  9 files changed, 255 insertions(+), 59 deletions(-)
> 
> -- 
> 2.39.0