Re: [Qemu-devel] removing on-demand msix vector allocation

[Michael S. Tsirkin]

On Mon, Dec 10, 2012 at 10:39:39AM +0100, Jan Kiszka wrote:
> On 2012-12-10 10:36, Michael S. Tsirkin wrote:
> > On Fri, Dec 07, 2012 at 08:37:22AM +0100, Jan Kiszka wrote:
> >> On 2012-12-06 08:59, Michael S. Tsirkin wrote:
> >>> I've been looking at handling of msix masking
> >>> in qemu. It looks like all of virtio,vfio and
> >>> device assignment implemented their own
> >>> similar but slightly different thing.
> >>> So I am inclined to move this handling to common
> >>> code in msix.c, adding irqfd support right there.
> >>>
> >>> While doing this rework, one of the more painful
> >>> bits of code to change is the code that dynamically
> >>> allocates msix table entries as we inject msi.
> >>> If this actually triggers it's going to be
> >>> painfully slow as route changes are rcu
> >>> write side in kernel.
> >>> Since recent kernels support direct injection,
> >>> do we care anymore? I think if you run out of
> >>> vectors, it's better to simply disable irqchip
> >>> than try to limp along changing routes all the time.
> >>
> >> But how would the logic without dynamic allocation look like? Always
> >> configure a route in the PCI layer if an MSI/MSI-X entry is enabled?
> >> That would also affect emulated devices that don't use irqfd, thus you
> >> would waste routing entries.
> > 
> > Yes. 
> > So we can fail during initialization and ask user to
> > disable irqchip: at the moment, at least in my testing,
> > dynamic swap out of MSI entries performs very badly
> > anyway.
> 
> That would be a poor approach as it regresses needlessly even over
> latest kernels.
> We only allocate/flush dynamically over older kernels
> without direct MSI injections.
> 
> What we need is a flag, set e.g. on msi[x]_init, to give the core a hint
> if it should allocate static routes for irqfd or if it will be able to
> use direct injection later on. Then we can simply do static allocation
> unconditionally on kernels without direct injection.
> 
> Jan
> 
> 

Makes sense.

-- 
MST