Re: [Qemu-devel] [PATCH v4 29/29] postcopy shared docs

[Marc-André Lureau]

On Thu, Mar 8, 2018 at 8:58 PM, Dr. David Alan Gilbert (git)
<address@hidden> wrote:
> From: "Dr. David Alan Gilbert" <address@hidden>
>
> Add some notes to the migration documentation for shared memory
> postcopy.
>
> Signed-off-by: Dr. David Alan Gilbert <address@hidden>


Reviewed-by: Marc-André Lureau <address@hidden>

> ---
>  docs/devel/migration.rst | 41 +++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 41 insertions(+)
>
> diff --git a/docs/devel/migration.rst b/docs/devel/migration.rst
> index 9d1b7657f0..e32b087f6e 100644
> --- a/docs/devel/migration.rst
> +++ b/docs/devel/migration.rst
> @@ -577,3 +577,44 @@ Postcopy now works with hugetlbfs backed memory:
>       hugepages works well, however 1GB hugepages are likely to be problematic
>       since it takes ~1 second to transfer a 1GB hugepage across a 10Gbps 
> link,
>       and until the full page is transferred the destination thread is 
> blocked.
> +
> +Postcopy with shared memory
> +---------------------------
> +
> +Postcopy migration with shared memory needs explicit support from the other
> +processes that share memory and from QEMU. There are restrictions on the 
> type of
> +memory that userfault can support shared.
> +
> +The Linux kernel userfault support works on `/dev/shm` memory and on 
> `hugetlbfs`
> +(although the kernel doesn't provide an equivalent to 
> `madvise(MADV_DONTNEED)`
> +for hugetlbfs which may be a problem in some configurations).
> +
> +The vhost-user code in QEMU supports clients that have Postcopy support,
> +and the `vhost-user-bridge` (in `tests/`) and the DPDK package have changes
> +to support postcopy.
> +
> +The client needs to open a userfaultfd and register the areas
> +of memory that it maps with userfault.  The client must then pass the
> +userfaultfd back to QEMU together with a mapping table that allows
> +fault addresses in the clients address space to be converted back to
> +RAMBlock/offsets.  The client's userfaultfd is added to the postcopy
> +fault-thread and page requests are made on behalf of the client by QEMU.
> +QEMU performs 'wake' operations on the client's userfaultfd to allow it
> +to continue after a page has arrived.
> +
> +.. note::
> +  There are two future improvements that would be nice:
> +    a) Some way to make QEMU ignorant of the addresses in the clients
> +       address space
> +    b) Avoiding the need for QEMU to perform ufd-wake calls after the
> +       pages have arrived
> +
> +Retro-fitting postcopy to existing clients is possible:
> +  a) A mechanism is needed for the registration with userfault as above,
> +     and the registration needs to be coordinated with the phases of
> +     postcopy.  In vhost-user extra messages are added to the existing
> +     control channel.
> +  b) Any thread that can block due to guest memory accesses must be
> +     identified and the implication understood; for example if the
> +     guest memory access is made while holding a lock then all other
> +     threads waiting for that lock will also be blocked.
> --
> 2.14.3
>
>



-- 
Marc-André Lureau