[Qemu-devel] [PATCH V8 0/9] Support more virtio queues

[Jason Wang]

We current limit the max virtio queues to 64. This is not sufficient
to support multiqueue devices (e.g recent Linux support up to 256
tap queues). So this series tries to let virtio to support more
queues.

No much works need to be done except:

- Increase the limit from 64 to 1024
- Let device_plugged() fail for virtio-ccw and s390-virtio when trying
  to plug virtio device with more than 64 virt queues. Since they
  currently don't support this.

With this patch, we can support up to 256 queues. Since x86 can only
allow about 240 interrupt vectors for MSI-X, current Linux driver can
only have about 80 queue pairs has their private MSI-X interrupt
vectors. With sharing IRQ with queue pairs (RFC posted in
https://lkml.org/lkml/2014/12/25/169), Linux driver can have up
to about 186 queue pairs has their private MSI-X interrupt vectors.

Test:
- Stress test on virtio-pci with x86_64-softmmu.
- Test s390-virtio and virtio-ccw can't start a virtio-net with more
  than 64 virtqueues.
- Compile test for other

Cc: Amit Shah <address@hidden>
Cc: Paolo Bonzini <address@hidden>
Cc: Alexander Graf <address@hidden>
Cc: Richard Henderson <address@hidden>
Cc: Cornelia Huck <address@hidden>
Cc: Christian Borntraeger <address@hidden>

Please review

Thanks

Changes from V7:
- Remove the patches that has been accepted.
- Adding all virtqueues in .realize() for virtio-net
- Let device_plugged() can fail
- Incrase the global limit from 64 to 1024 while let virtio-ccw and
  s390-virtio's device_plugged() fail when the number of queues is
  greater than 64
- Drop the migration capability stuffs.

Changes from V6:
- Tweak some commit logs
- Replace the missed VIRTIO_PCI_MAX in ccw with a bus specific limit
- Bump the pci queue limit from 513 to 1024

Changes from V5:
- Rebase to HEAD
- Drop the patch that introduce helper to get vq index since there has
  already one
- Don't try to solve the migration compatibility issue for invalid
  configuration like vectors > 128 with only 64 virto queues. This is
  done by let msix calaucalte the correct bar size and drop the patch
  18 that intrdouces the auto_msix_bar_size.
- Cleanup the bar size and pba offset calcuation code
- Add a comment to explain the new virtio pci queue limit number 513

Changes from V4:
- Drop the patch of bus limitation validation and send it as an
  independent patch
- Don't let the virtio-pci to calculate the MSI-X bar size, instead,
  tell the msix code whether or not to use legacy bar size and let it
  to do this
- Make sure the MSI-X bar size is at least 4K
- Subject typos fixes for patch 8
- Mention adapter routes changes in the commit log of patch 9
- Only use the vector to queue list for virtio-pci to avoid the
  unnecessary possible overhead on other transport.

Changes from V3:
- rebase to master and target to 2.4
- handling compat issues for spapr
- fixes for hmp command completion when we have a nic with 256 queues
- using VIRTIO_NO_VECTOR instead of 0 for invalid virtqueue in
  virtio-ccw
- fix compile issues for ppc64-softmmu
- don't export VIRTIO_CCW_QUEUE_MAX by introducing a gsi limit and
  inherit in ccw
- use transport specific queue limit in virtio-serial
- correct the stale comment for AdapterRoutes and move it to ccw patch
- replace 128 with queue_max + 64 and add a comment for this in
  virtio_ccw_notify()

Changes from V2:
- move transport specific limitation to their implementation. (The
  left is VIRTIO_CCW_QUEUE_MAX since I fail to find a common header
  files other than virtio.h)
- use virtio_get_queue_ma) if possible in virtio.c
- AdapterRoutes should use ccw limit
- introduce a vector to queue mapping for virito devices and speedup
  the MSI-X masking and unmasking through this.

Changes from V1:
- add a validation against the bus limitation
- switch to use a bus specific queue limit instead of a global one,
  this will allow us to just increase the limit of one transport
  without disturbing others.
- only increase the queue limit of virtio-pci
- limit the maximum number of virtio queues to 64 for legacy machine
  types

Jason Wang (9):
  virtio-net: adding all queues in .realize()
  virtio: device_plugged() can fail
  virtio: introduce virtio_get_num_queues()
  virtio-ccw: introduce ccw specific queue limit
  virtio-ccw: validate the number of queues against bus limitation
  virtio-s390: introduce virito s390 queue limit
  virtio-s390: introduce virtio_s390_device_plugged()
  virtio: rename VIRTIO_PCI_QUEUE_MAX to VIRTIO_QUEUE_MAX
  virtio: increase the queue limit to 1024

 hw/char/virtio-serial-bus.c    |  2 +-
 hw/net/virtio-net.c            | 63 +++++++++++++-----------------------------
 hw/s390x/s390-virtio-bus.c     | 20 ++++++++++++--
 hw/s390x/s390-virtio-ccw.c     |  2 +-
 hw/s390x/virtio-ccw.c          | 23 ++++++++++-----
 hw/scsi/virtio-scsi.c          |  4 +--
 hw/virtio/virtio-bus.c         |  6 ++--
 hw/virtio/virtio-mmio.c        |  6 ++--
 hw/virtio/virtio-pci.c         | 12 ++++----
 hw/virtio/virtio.c             | 46 ++++++++++++++++++++----------
 include/hw/s390x/s390_flic.h   |  5 +++-
 include/hw/virtio/virtio-bus.h |  4 +--
 include/hw/virtio/virtio.h     |  3 +-
 13 files changed, 108 insertions(+), 88 deletions(-)

-- 
2.1.4