[Qemu-devel] Re: [PATCH] Inter-VM shared memory PCI device

[Cam Macdonell]

On Tue, Mar 9, 2010 at 6:03 AM, Avi Kivity <address@hidden> wrote:
> On 03/09/2010 02:49 PM, Arnd Bergmann wrote:
>>
>> On Monday 08 March 2010, Cam Macdonell wrote:
>>
>>>
>>> enum ivshmem_registers {
>>>     IntrMask = 0,
>>>     IntrStatus = 2,
>>>     Doorbell = 4,
>>>     IVPosition = 6,
>>>     IVLiveList = 8
>>> };
>>>
>>> The first two registers are the interrupt mask and status registers.
>>> Interrupts are triggered when a message is received on the guest's
>>> eventfd from
>>> another VM.  Writing to the 'Doorbell' register is how synchronization
>>> messages
>>> are sent to other VMs.
>>>
>>> The IVPosition register is read-only and reports the guest's ID number.
>>>  The
>>> IVLiveList register is also read-only and reports a bit vector of
>>> currently
>>> live VM IDs.
>>>
>>> The Doorbell register is 16-bits, but is treated as two 8-bit values.
>>>  The
>>> upper 8-bits are used for the destination VM ID.  The lower 8-bits are
>>> the
>>> value which will be written to the destination VM and what the guest
>>> status
>>> register will be set to when the interrupt is trigger is the destination
>>> guest.
>>> A value of 255 in the upper 8-bits will trigger a broadcast where the
>>> message
>>> will be sent to all other guests.
>>>
>>
>> This means you have at least two intercepts for each message:
>>
>> 1. Sender writes to doorbell
>> 2. Receiver gets interrupted
>>
>> With optionally two more intercepts in order to avoid interrupting the
>> receiver every time:
>>
>> 3. Receiver masks interrupt in order to process data
>> 4. Receiver unmasks interrupt when it's done and status is no longer
>> pending
>>
>> I believe you can do much better than this, you combine status and mask
>> bits, making this level triggered, and move to a bitmask of all guests:
>>
>> In order to send an interrupt to another guest, the sender first checks
>> the bit for the receiver. If it's '1', no need for any intercept, the
>> receiver will come back anyway. If it's zero, write a '1' bit, which
>> gets OR'd into the bitmask by the host. The receiver gets interrupted
>> at a raising edge and just leaves the bit on, until it's done processing,
>> then turns the bit off by writing a '1' into its own location in the mask.
>>
>
> We could make the masking in RAM, not in registers, like virtio, which would
> require no exits.  It would then be part of the application specific
> protocol and out of scope of of this spec.
>

This kind of implementation would be possible now since with UIO it's
up to the application whether to mask interrupts or not and what
interrupts mean.  We could leave the interrupt mask register for those
who want that behaviour.  Arnd's idea would remove the need for the
Doorbell and Mask, but we will always need at least one MMIO register
to send whatever interrupts we do send.

Cam