[Qemu-devel] Re: [PATCH v3 0/2] Inter-VM shared memory PCI device

[Avi Kivity]

On 03/25/2010 08:17 PM, Cam Macdonell wrote:
> > > I had a hunch it was probably considered.  That explains why irqfd
> > > doesn't have a datamatch field.  I guess supporting multiple MSI
> > > vectors with one doorbell per guest isn't possible if one 1 bit of
> > > information can be communicated.
> > >
> > >        
> > Actually you can have one doorbell supporting multiple vectors and guests,
> > simply divide the data value into two bit fields, one for the vector and one
> > for the guest.  A single write gets both values into the host, which can
> > then use datamatch to trigger the correct eventfd (which is wired to an
> > irqfd in another guest).
> >      
> At 4-bits per guest, a single write is then limited to 8 guests (with
> 32-bit registers), we could got to 64-bit.
>    

I meant a unicast doorbell: 16 bits for guest ID, 16 bits for vector number.

> >      
> > > So, ioeventfd/irqfd restricts MSI to 1 vector between guests.  Should
> > > multi-MSI even be supported then in the non-ioeventfd/irq case?
> > > Otherwise ioeventfd/irqfd become more than an implementation detail.
> > >
> > >        
> > I lost you.  Please re-explain.
> >      
> An irqfd can only trigger a single vector in a guest.  Right now I
> only have one eventfd per guest.    So ioeventfd/irqfd restricts the
> current implementation to a single vector that a guest can trigger.
> Without irqfd, eventfds can be used like registers a write the number
> of the vector they want to trigger, but as you point out it is racy.
>    

You can't use eventfds as registers.  The next write will add to the 
current value.

> So, supporting multiple vectors via irqfd requires multiple eventfds
> for each guest (one per vector).   a total of (# of guests) X (# of
> vectors) are required.  If we're limited to 8 or 16 guests that's not
> too bad, but since the server opens them all we're restricted to 1024,
> but that's a pretty high ceiling for this purpose.
>    

I'm sure we can raise the fd ulimit for this.  Note, I think qemus need 
the ulimit raised as well, since an fd passed via SCM_RIGHTS probably 
counts as an open file.

--
Do not meddle in the internals of kernels, for they are subtle and quick to 
panic.