Re: RFC: New device for zero-copy VM memory access

[Marc-André Lureau]

Hi

On Mon, Nov 4, 2019 at 4:05 PM <address@hidden> wrote:
>
>
>
> On 2019-11-04 22:55, Dr. David Alan Gilbert wrote:
> > * address@hidden (address@hidden) wrote:
> >>
> >>
> >> On 2019-11-03 21:10, address@hidden wrote:
> >> > On 2019-11-01 02:52, Dr. David Alan Gilbert wrote:
> >> > > * address@hidden (address@hidden) wrote:
> >> > > >
> >> > > >
> >> > > > On 2019-11-01 01:52, Peter Maydell wrote:
> >> > > > > On Thu, 31 Oct 2019 at 14:26, <address@hidden> wrote:
> >> > > > > > As the author of Looking Glass, I also have to consider the
> >> > > > > > maintenance
> >> > > > > > and the complexity of implementing the vhost protocol into the
> >> > > > > > project.
> >> > > > > > At this time a complete Porthole client can be implemented in 150
> >> > > > > > lines
> >> > > > > > of C without external dependencies, and most of that is 
> >> > > > > > boilerplate
> >> > > > > > socket code. This IMO is a major factor in deciding to avoid
> >> > > > > > vhost-user.
> >> > > > >
> >> > > > > This is essentially a proposal that we should make our project and
> >> > > > > code more complicated so that your project and code can be simpler.
> >> > > > > I hope you can see why this isn't necessarily an argument that 
> >> > > > > will hold
> >> > > > > very much weight for us :-)
> >> > > >
> >> > > > Certainly, I do which is why I am still going to see about using
> >> > > > vhost,
> >> > > > however, a device that uses vhost is likely more complex then
> >> > > > the device
> >> > > > as it stands right now and as such more maintenance would be
> >> > > > involved on
> >> > > > your end also. Or have I missed something in that vhost-user can
> >> > > > be used
> >> > > > directly as a device?
> >> > >
> >> > > The basic vhost-user stuff isn't actually that hard;  if you aren't
> >> > > actually shuffling commands over the queues you should find it pretty
> >> > > simple - so I think your assumption about it being simpler if you
> >> > > avoid
> >> > > it might be wrong.  It might be easier if you use it!
> >> >
> >> > I have been looking into this and I am yet to find some decent
> >> > documentation or a simple device example I can use to understand how to
> >> > create such a device. Do you know of any reading or examples I can
> >> > obtain
> >> > on how to get an initial do nothing device up and running?
> >> >
> >> > -Geoff
> >>
> >> Scratch that, the design just solidified for me and I am now making
> >> progress, however it seems that vhost-user can't do what we need here:
> >>
> >> 1) I dont see any way to recieve notification of socket disconnection,
> >> in
> >> our use case the client app needs to be able to be (re)connected
> >> dynamically. It might be possible to get this event by registering it
> >> on
> >> the chardev manually but this seems like it would be a kludge.
> >
> > My understanding was that someone added support for reconnection of
> > vhost-user;  I'm not sure of the detail - cc'ing in Maxime and
> > Marc-Andre.
> >
> >> 2) I don't see any method of notifying the vhost-user client of the
> >> removal of a shared memory mapping. Again, these may not be
> >> persistently
> >> mapped in the guest as we have no control over the buffer allocation,
> >> and
> >> as such, we need a method to notify the client that the mapping has
> >> become
> >> invalid.
> >>
> >> 3) VHOST_USER_SET_MEM_TABLE is a one time request, again this breaks
> >> our
> >> usage as we need to change this dynamically at runtime.
> >
> > I've seen (3) being sent multiple times (It's messy but it happens); so
> > I think that fixes (2) as well for you.
>
> Yes, but it's ignored.
>
>      /*
>       * For non-vring specific requests, like VHOST_USER_SET_MEM_TABLE,
>       * we just need send it once in the first time. For later such
>       * request, we just ignore it.
>       */
>      if (vhost_user_one_time_request(msg->hdr.request) && dev->vq_index
> != 0) {
>           msg->hdr.flags &= ~VHOST_USER_NEED_REPLY_MASK;
>           return 0;
>      }

This code was added to avoid sending the same mapping for each queue:
https://github.com/qemu/qemu/commit/b931bfbf042983f311b3b09894d8030b2755a638
(arguably, the filtering could have been done earlier)

But if you reconnect, it should still send it again at least once (for vq #0).

vhost-user-bridge reconnect used to work quite reliably, I haven't
tested recently.


>
> >
> > Dave
> >
> >> Unless there are viable solutions to these problems there is no way
> >> that
> >> vhost-user can be used for this kind of a device.
> >>
> >> -Geoff
> >>
> >> >
> >> > >
> >> > > Dave
> >> > >
> >> > > > >
> >> > > > > thanks
> >> > > > > -- PMM
> >> > > --
> >> > > Dr. David Alan Gilbert / address@hidden / Manchester, UK
> > --
> > Dr. David Alan Gilbert / address@hidden / Manchester, UK