[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH 18/21] vfio-pci: Make host MSI-X enable track guest
From: |
Michael Roth |
Subject: |
[Qemu-devel] [PATCH 18/21] vfio-pci: Make host MSI-X enable track guest |
Date: |
Wed, 16 Jan 2013 10:49:20 -0600 |
From: Alex Williamson <address@hidden>
Guests typically enable MSI-X with all of the vectors in the MSI-X
vector table masked. Only when the vector is enabled does the vector
get unmasked, resulting in a vector_use callback. These two points,
enable and unmask, correspond to pci_enable_msix() and request_irq()
for Linux guests. Some drivers rely on VF/PF or PF/fw communication
channels that expect the physical state of the device to match the
guest visible state of the device. They don't appreciate lazily
enabling MSI-X on the physical device.
To solve this, enable MSI-X with a single vector when the MSI-X
capability is enabled and immediate disable the vector. This leaves
the physical device in exactly the same state between host and guest.
Furthermore, the brief gap where we enable vector 0, it fires into
userspace, not KVM, so the guest doesn't get spurious interrupts.
Ideally we could call VFIO_DEVICE_SET_IRQS with the right parameters
to enable MSI-X with zero vectors, but this will currently return an
error as the Linux MSI-X interfaces do not allow it.
Signed-off-by: Alex Williamson <address@hidden>
Cc: address@hidden
(cherry picked from commit b0223e29afdc88cc262a764026296414396cd129)
Signed-off-by: Michael Roth <address@hidden>
---
hw/vfio_pci.c | 30 ++++++++++++++++++++++++++----
1 file changed, 26 insertions(+), 4 deletions(-)
diff --git a/hw/vfio_pci.c b/hw/vfio_pci.c
index fbfe670..46dce55 100644
--- a/hw/vfio_pci.c
+++ b/hw/vfio_pci.c
@@ -562,8 +562,8 @@ static int vfio_enable_vectors(VFIODevice *vdev, bool msix)
return ret;
}
-static int vfio_msix_vector_use(PCIDevice *pdev,
- unsigned int nr, MSIMessage msg)
+static int vfio_msix_vector_do_use(PCIDevice *pdev, unsigned int nr,
+ MSIMessage *msg, IOHandler *handler)
{
VFIODevice *vdev = DO_UPCAST(VFIODevice, pdev, pdev);
VFIOMSIVector *vector;
@@ -587,7 +587,7 @@ static int vfio_msix_vector_use(PCIDevice *pdev,
* Attempt to enable route through KVM irqchip,
* default to userspace handling if unavailable.
*/
- vector->virq = kvm_irqchip_add_msi_route(kvm_state, msg);
+ vector->virq = msg ? kvm_irqchip_add_msi_route(kvm_state, *msg) : -1;
if (vector->virq < 0 ||
kvm_irqchip_add_irqfd_notifier(kvm_state, &vector->interrupt,
vector->virq) < 0) {
@@ -596,7 +596,7 @@ static int vfio_msix_vector_use(PCIDevice *pdev,
vector->virq = -1;
}
qemu_set_fd_handler(event_notifier_get_fd(&vector->interrupt),
- vfio_msi_interrupt, NULL, vector);
+ handler, NULL, vector);
}
/*
@@ -639,6 +639,12 @@ static int vfio_msix_vector_use(PCIDevice *pdev,
return 0;
}
+static int vfio_msix_vector_use(PCIDevice *pdev,
+ unsigned int nr, MSIMessage msg)
+{
+ return vfio_msix_vector_do_use(pdev, nr, &msg, vfio_msi_interrupt);
+}
+
static void vfio_msix_vector_release(PCIDevice *pdev, unsigned int nr)
{
VFIODevice *vdev = DO_UPCAST(VFIODevice, pdev, pdev);
@@ -697,6 +703,22 @@ static void vfio_enable_msix(VFIODevice *vdev)
vdev->interrupt = VFIO_INT_MSIX;
+ /*
+ * Some communication channels between VF & PF or PF & fw rely on the
+ * physical state of the device and expect that enabling MSI-X from the
+ * guest enables the same on the host. When our guest is Linux, the
+ * guest driver call to pci_enable_msix() sets the enabling bit in the
+ * MSI-X capability, but leaves the vector table masked. We therefore
+ * can't rely on a vector_use callback (from request_irq() in the guest)
+ * to switch the physical device into MSI-X mode because that may come a
+ * long time after pci_enable_msix(). This code enables vector 0 with
+ * triggering to userspace, then immediately release the vector, leaving
+ * the physical device with no vectors enabled, but MSI-X enabled, just
+ * like the guest view.
+ */
+ vfio_msix_vector_do_use(&vdev->pdev, 0, NULL, NULL);
+ vfio_msix_vector_release(&vdev->pdev, 0);
+
if (msix_set_vector_notifiers(&vdev->pdev, vfio_msix_vector_use,
vfio_msix_vector_release)) {
error_report("vfio: msix_set_vector_notifiers failed\n");
--
1.7.9.5
- [Qemu-devel] [PATCH 12/21] qxl: save qemu_create_displaysurface_from result, (continued)
- [Qemu-devel] [PATCH 12/21] qxl: save qemu_create_displaysurface_from result, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 10/21] pixman: fix vnc tight png/jpeg support, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 08/21] seabios: update to e8a76b0f225bba5ba9d63ab227e0a37b3beb1059, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 13/21] qxl+vnc: register a vm state change handler for dummy spice_server, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 15/21] migration: fix migration_bitmap leak, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 16/21] buffered_file: do not send more than s->bytes_xfer bytes per tick, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 20/21] vfio-pci: Loosen sanity checks to allow future features, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 21/21] raw-posix: fix bdrv_aio_ioctl, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 09/21] Update seabios to a810e4e72a0d42c7bc04eda57382f8e019add901, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 19/21] pci-assign: Enable MSIX on device to match guest, Michael Roth, 2013/01/16
- [Qemu-devel] [PATCH 18/21] vfio-pci: Make host MSI-X enable track guest,
Michael Roth <=
- [Qemu-devel] [PATCH 14/21] e1000: Discard oversized packets based on SBP|LPE, Michael Roth, 2013/01/16
- Re: [Qemu-devel] [Qemu-stable] Patch Round-up for stable 1.3.1, freeze Monday, Michael Tokarev, 2013/01/16
- [Qemu-devel] [PATCH 17/21] target-xtensa: fix search_pc for the last TB opcode, Michael Roth, 2013/01/16
- Re: [Qemu-devel] [Qemu-stable] Patch Round-up for stable 1.3.1, freeze Monday, Doug Goldstein, 2013/01/18