Re: [Qemu-discuss] xHCI controller/USB passthrough device - event ring full error on high packet rate

[Samuel Brian]

Hi Gerd,

In the guest, the qmi_wwan kernel driver drives the modem. It seems to 
behave properly with respect to interrupts -- the rate of interrupts I 
see in /proc/interrupts is not proportional to the packet rate.

I've tried a few things to help the performance:
 1. pinning qemu and the vcpu to separate cores on the hypervisor (that 
no other processes use)
 2. forcing the USB device to use USB 2.0 and attach it to a virtual 
EHCI controller

Both help, (especially 2 which prevented some xHCI controller problems) 
but the maximum packet rates I could achieve still weren't great.

> Probably the event ring is _really_ full.  The linux kernel in the guest
> probably can't unqueue the events fast enough due to virtualization
> overhead and/or the vcpu not being scheduled for a while.

I think that just is the case.
For now I've gone off a VM solution and am pursuing a container solution 
to avoid the libusb overhead. I'm sure I'll circle back around to qemu 
again soon though :D

Thanks for your help,

Sam


On 01/02/18 20:38, Gerd Hoffmann wrote:
> On Thu, Dec 07, 2017 at 05:52:25PM +1000, Samuel Brian wrote:
> > Hi all,
>
> Sorry, for the long delay, was offline for a few weeks due to a broken
> finger and now I'm going back in my inbox looking for unanswered mails.
>
> >    xhci_hcd 0000:00:05.0: ERROR unknown event type 37
>
> > Qemu gives me this error message before it sends that event type 37
> > (CC_EVENT_RING_FULL_ERROR):
>
> Seems linux simply hasn't implemented that ...
>
> > I think my problem is sort of mentioned in
> > http://lists.nongnu.org/archive/html/qemu-devel/2017-02/msg00598.html
> >
> > > It does suspious ERDP updates now and
> > > then (not every time, seems we must hit a race window for this to
> > > happen), which in turn makes the qemu xhci emulation think the event
> > > ring is full.  Things go south from here ...
> >
> > I suppose I'm just really good at hitting that race window
>
> I don't think so.  If you are running latest qemu (2.11, I think 2.10 is
> new enough too) this should be fixed in qemu.  If you are running
> something older try upgrading.
>
> > -- I can udpblast
> > at 130,000 network packets per second over the modem when not running in a
> > VM.
>
> Probably the event ring is _really_ full.  The linux kernel in the guest
> probably can't unqueue the events fast enough due to virtualization
> overhead and/or the vcpu not being scheduled for a while.
>
> > For reference, I'm using Qemu v2.10.1 and libusb 1.0.21 and running the VM
> > on a Linux 4.14 host with:
>
> Who drives the modem in the guest?  libusb too?  Or some kernel driver?
>
> I would try to reduce the number of events, by asking xhci for an
> interrupt on every 10th transfer, so the irq rate in the guest goes down
> from 130000 to 13000.  Not sure there is an easy way to do so though.
>
> cheers,
>    Gerd