[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [PATCH v3 00/29] postcopy+vhost-user/shared ram
From: |
Dr. David Alan Gilbert |
Subject: |
Re: [Qemu-devel] [PATCH v3 00/29] postcopy+vhost-user/shared ram |
Date: |
Wed, 28 Feb 2018 18:38:21 +0000 |
User-agent: |
Mutt/1.9.2 (2017-12-15) |
* Michael S. Tsirkin (address@hidden) wrote:
> On Tue, Feb 27, 2018 at 08:05:25PM +0000, Dr. David Alan Gilbert wrote:
> > * Michael S. Tsirkin (address@hidden) wrote:
> > > On Fri, Feb 16, 2018 at 01:15:56PM +0000, Dr. David Alan Gilbert (git)
> > > wrote:
> > > > From: "Dr. David Alan Gilbert" <address@hidden>
> > > >
> > > > Hi,
> > > > This is the first non-RFC version of this patch set that
> > > > enables postcopy migration with shared memory to a vhost user process.
> > > > It's based off current head.
> > > >
> > > > I've tested with vhost-user-bridge and a modified dpdk; both very
> > > > lightly.
> > > >
> > > > Compared to v2 we're now using the just-merged reworks to the vhost
> > > > code (suggested by Igor), so that the huge page region merging is now a
> > > > lot simpler
> > > > in this series. The handshake between the client and the qemu for the
> > > > set-mem-table is now a bit more complex to resolve a previous race where
> > > > the client would start sending requests to the qemu prior to the qemu
> > > > being ready to accept them.
> > > >
> > > > Dave
> > >
> > > From vhost-user POV this seems mostly fine to me.
> >
> > OK, great - it would be nice to get this merged in the upcoming release
> > (Hint: Anyone else please review!)
> >
> > > I would like to have dependency of specific messages on the
> > > protocol features documented, and the order of messages
> > > documented a bit more explicitly.
> >
> > Something like the following? (appropriately merged in with the
> > individual commits):
> >
> > diff --git a/docs/interop/vhost-user.txt b/docs/interop/vhost-user.txt
> > index 4bf7d8ef99..7841812766 100644
> > --- a/docs/interop/vhost-user.txt
> > +++ b/docs/interop/vhost-user.txt
> > @@ -461,7 +461,7 @@ Master message types
> > for each memory mapped region. The size and ordering of the fds
> > matches
> > the number and ordering of memory regions.
> >
> > - When postcopy-listening has been received, SET_MEM_TABLE replies with
> > + When VHOST_USER_POSTCOPY_LISTEN has been received, SET_MEM_TABLE
> > replies with
> > the bases of the memory mapped regions to the master. It must have
> > mmap'd
> > the regions but not yet accessed them and should not yet generate a
> > userfault
> > event. Note NEED_REPLY_MASK is not set in this case.
> > @@ -687,7 +687,8 @@ Master message types
> > Master payload: N/A
> > Slave payload: userfault fd + u64
> >
> > - Master advises slave that a migration with postcopy enabled is
> > underway,
> > + When VHOST_USER_PROTOCOL_F_PAGEFAULT is supported, the
> > + master advises slave that a migration with postcopy enabled is
> > underway,
> > the slave must open a userfaultfd for later use.
> > Note that at this stage the migration is still in precopy mode.
> >
> > @@ -696,6 +697,8 @@ Master message types
> > Master payload: N/A
> >
> > Master advises slave that a transition to postcopy mode has happened.
> > + This is always sent sometime after a VHOST_USER_POSTCOPY_ADVISE, and
> > + thus only when VHOST_USER_PROTOCOL_F_PAGEFAULT is supported.
> >
> > * VHOST_USER_POSTCOPY_END
> > Id: 28
> > @@ -704,6 +707,8 @@ Master message types
> > Master advises that postcopy migration has now completed. The
> > slave must disable the userfaultfd. The response is an
> > acknowledgement
> > only.
> > + This message is sent at the end of the migration, after
> > + VHOST_USER_POSTCOPY_LISTEN was previously sent.
>
> And maybe mention VHOST_USER_PROTOCOL_F_PAGEFAULT here too.
Done.
Dave
> > Slave message types
> > -------------------
> >
> > Dave
> >
> > >
> > >
> > >
> > > > Dr. David Alan Gilbert (29):
> > > > migrate: Update ram_block_discard_range for shared
> > > > qemu_ram_block_host_offset
> > > > postcopy: use UFFDIO_ZEROPAGE only when available
> > > > postcopy: Add notifier chain
> > > > postcopy: Add vhost-user flag for postcopy and check it
> > > > vhost-user: Add 'VHOST_USER_POSTCOPY_ADVISE' message
> > > > libvhost-user: Support sending fds back to qemu
> > > > libvhost-user: Open userfaultfd
> > > > postcopy: Allow registering of fd handler
> > > > vhost+postcopy: Register shared ufd with postcopy
> > > > vhost+postcopy: Transmit 'listen' to client
> > > > postcopy+vhost-user: Split set_mem_table for postcopy
> > > > migration/ram: ramblock_recv_bitmap_test_byte_offset
> > > > libvhost-user+postcopy: Register new regions with the ufd
> > > > vhost+postcopy: Send address back to qemu
> > > > vhost+postcopy: Stash RAMBlock and offset
> > > > vhost+postcopy: Send requests to source for shared pages
> > > > vhost+postcopy: Resolve client address
> > > > postcopy: wake shared
> > > > postcopy: postcopy_notify_shared_wake
> > > > vhost+postcopy: Add vhost waker
> > > > vhost+postcopy: Call wakeups
> > > > libvhost-user: mprotect & madvises for postcopy
> > > > vhost-user: Add VHOST_USER_POSTCOPY_END message
> > > > vhost+postcopy: Wire up POSTCOPY_END notify
> > > > vhost: Huge page align and merge
> > > > postcopy: Allow shared memory
> > > > libvhost-user: Claim support for postcopy
> > > > postcopy shared docs
> > > >
> > > > contrib/libvhost-user/libvhost-user.c | 303 ++++++++++++++++++++++++-
> > > > contrib/libvhost-user/libvhost-user.h | 8 +
> > > > docs/devel/migration.rst | 41 ++++
> > > > docs/interop/vhost-user.txt | 42 ++++
> > > > exec.c | 85 +++++--
> > > > hw/virtio/trace-events | 16 +-
> > > > hw/virtio/vhost-user.c | 411
> > > > +++++++++++++++++++++++++++++++++-
> > > > hw/virtio/vhost.c | 66 +++++-
> > > > include/exec/cpu-common.h | 4 +
> > > > migration/migration.c | 6 +
> > > > migration/migration.h | 4 +
> > > > migration/postcopy-ram.c | 350
> > > > +++++++++++++++++++++++------
> > > > migration/postcopy-ram.h | 69 ++++++
> > > > migration/ram.c | 5 +
> > > > migration/ram.h | 1 +
> > > > migration/savevm.c | 13 ++
> > > > migration/trace-events | 6 +
> > > > trace-events | 3 +-
> > > > vl.c | 2 +
> > > > 19 files changed, 1337 insertions(+), 98 deletions(-)
> > > >
> > > > --
> > > > 2.14.3
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK
- [Qemu-devel] [PATCH v3 27/29] postcopy: Allow shared memory, (continued)
- [Qemu-devel] [PATCH v3 27/29] postcopy: Allow shared memory, Dr. David Alan Gilbert (git), 2018/02/16
- [Qemu-devel] [PATCH v3 28/29] libvhost-user: Claim support for postcopy, Dr. David Alan Gilbert (git), 2018/02/16
- [Qemu-devel] [PATCH v3 23/29] libvhost-user: mprotect & madvises for postcopy, Dr. David Alan Gilbert (git), 2018/02/16
- [Qemu-devel] [PATCH v3 24/29] vhost-user: Add VHOST_USER_POSTCOPY_END message, Dr. David Alan Gilbert (git), 2018/02/16
- [Qemu-devel] [PATCH v3 29/29] postcopy shared docs, Dr. David Alan Gilbert (git), 2018/02/16
- Re: [Qemu-devel] [PATCH v3 00/29] postcopy+vhost-user/shared ram, Michael S. Tsirkin, 2018/02/27