Re: [Qemu-devel] [QA-virtio]:Why vring size is limited to 1024?

[Michael S. Tsirkin]

On Wed, Oct 08, 2014 at 01:59:13PM +0300, Avi Kivity wrote:
> 
> On 10/08/2014 01:55 PM, Michael S. Tsirkin wrote:
> >>>>Even more useful is getting rid of the desc array and instead passing 
> >>>>descs
> >>>>inline in avail and used.
> >>>You expect this to improve performance?
> >>>Quite possibly but this will have to be demonstrated.
> >>>
> >>The top vhost function in small packet workloads is vhost_get_vq_desc, and
> >>the top instruction within that (50%) is the one that reads the first 8
> >>bytes of desc.  It's a guaranteed cache line miss (and again on the guest
> >>side when it's time to reuse).
> >OK so basically what you are pointing out is that we get 5 accesses:
> >read of available head, read of available ring, read of descriptor,
> >write of used ring, write of used ring head.
> 
> Right.  And only read of descriptor is not amortized.
> 
> >If processing is in-order, we could build a much simpler design, with a
> >valid bit in the descriptor, cleared by host as descriptors are
> >consumed.
> >
> >Basically get rid of both used and available ring.
> 
> That only works if you don't allow reordering, which is never the case for
> block, and not the case for zero-copy net.  It also has writers on both side
> of the ring.
> 
> The right design is to keep avail and used, but instead of making them rings
> of pointers to descs, make them rings of descs.
> 
> The host reads descs from avail, processes them, then writes them back on
> used (possibly out-of-order).  The guest writes descs to avail and reads
> them back from used.
> 
> You'll probably have to add a 64-bit cookie to desc so you can complete
> without an additional lookup.

My old presentation from 2012 or so suggested something like this.
We don't need a 64 bit cookie I think - a small 16 bit one
should be enough.

> >
> >Sounds good in theory.
> >
> >>Inline descriptors will amortize the cache miss over 4 descriptors, and will
> >>allow the hardware to prefetch, since the descriptors are linear in memory.
> >If descriptors are used in order (as they are with current qemu)
> >then aren't they amortized already?
> >