[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PATCH v6 00/26] IOMMU: Enable interrupt remapping for Inte
From: |
Peter Xu |
Subject: |
[Qemu-devel] [PATCH v6 00/26] IOMMU: Enable interrupt remapping for Intel IOMMU |
Date: |
Thu, 5 May 2016 11:25:35 +0800 |
Hi, all,
This is v6 for Intel IOMMU IR support. This series introduced quite
a few new patches based on v5. Sorry for that (Yes, Jan is
contributing to it as well, though most of which are really good
ideas for me :). Hopefully we can get its convergence in this
version.
To make the review easier, I tried to keep all the existing patches
and indexes (also, this is easier for me too to do the
modifications, and logically I feel this make more sense and clean,
please let me know if I am wrong). Patches 1-18 are v5 patches, and
patches 19-26 are newly added patches.
All the new patches may need more review, many of them are outside
Intel IOMMU scope, and touching other part of codes, which I am
still not very sure about.
Testing is only covering basic smoke test for the following matrix:
- IR enabled/disable
- kernel irqchip off/split
- network device: tap with/without vhost, e1000
Here's the change log. Please review. Thanks,
v6 changes:
- patch 10: use write_with_attrs() rather than write(), preparing
for SID verification [Jan]
- patch 17-18: add r-b line from Radim [Radim]
- new patch 19: put together Jan's EIM patch [Jan]
- new patch 20: add SID validation process
- new patch 21-22: introduce X86IOMMU class, which is the parent of
IntelIOMMU class. Patch 21 only introduce the class and did
nothing, patch 22 cleaned up all the vtd_*() hooks into x86
ones. This is only a start. In the future, we can abstract more
things into X86IOMMU class, like iotlb, address spaces mgmt,
etc. [Jan]
- new patch 23-25: this is to do IEC notify to all irqfd consumers
like vhost/vfio. patch 23 changed interface for
kvm_irqchip_add_msi_route(), provide vector info rather than a raw
MSI message. Patch 24 added new hooks to do arch-specific
notification on addition/deletion of msi routes. Patch 25 is x86
specific, which added one more IEC notifier for msi routes. [Jan]
- new patch 26: this is to partially solve the issue that Jan has
encountered (1 sec delay when invalidating IR cache).
v5 changes:
- patch 10: add vector checking for IOAPIC interrupts (this may help
debug in the future, will only generate warning if specify
IOMMU_DEBUG)
- patch 13: replace error_report() with a trace. [Jan]
- patch 14: rename parameter "intr" to "intremap", to be aligned
with kernel parameter [Jan]
- patch 15: fix comments for vtd_iec_notify_fn
- patch 17 & 18 (added): fix issue when IR enabled with devices
using level-triggered interrupts, like e1000. Adding it to the end
of series, since this issue never happen without IR.
Patch 17 adds read-only check for IOAPIC entries.
Patch 18 clears remote IRR bit when entry configured as
edge-triggered.
v4 changes (all patch number corresponds to v3):
- add one patch at the start of v3 series: I missed to send the
first patch in v3. adding it in. [Jan]
- patch 9: add support for compatible mode (no reason not to support
it, if not, we will get some warnings when using split irqchip)
- patch 11: further simplify ioapic_update_kvm_routes() using the
helper function.
- patch 12: tweak on kvm_arch_fixup_msi_route() rather than
ioapic_update_kvm_routes() only. [Radim]
- add patch 15: introduce IEC (Interrupt Entry Cache) invalidation
notifier list. We can register to this list if we want to be
notified when we got IR invalidation requests [Radim]
- add patch 16: let IOAPIC the first consumer for the above IEC
notifier list. [Radim]
- several other trivial fixes (like moving some defines from .c to
.h, moving several lines of changes from one patch to another to
make it make more sense, etc.)
v3 changes (all patch numbers corresponds to v2):
- patch 1 (-> v3 patch 13)
- move to the end of series [Alex]
- patch 10 (dropped)
- drop this one, since re-worked on IOAPIC support, so we do not
need this any more.
- patch 12 (-> v3 patch 10)
- leverage MSI path for IOAPIC IR [Jan]
- patch 13 (v3 -> patch 9)
- remove vtd_interrupt_remap_msi() declaration by reordering the
functions [mst]
- vtd_generate_msi_message(): init msg using {}, remove FIXME
[mst]
- new patches
- v3 patch 11: introduce ioapic_entry_parse() helper function
- v3 patch 12: add support for kernel-irqchip=split. This needs
more reviews, logically this should enable lots of things:
splitted irqchip, irqfd, vhost, and irqfd support for
passthrough devices (not tested). Please refer to the patch for
more information.
v2 changes:
- patch 1
- rename "int_remap" to "intr" in several places [Marcel]
- remove "Intel" specific words in desc or commit message, prepare
itself with further AMD support [Marcel]
- avoid using object_property_get_bool() [Marcel]
- patch 5
- use PCI bus number 0xff rather than 0xf0 for the IOAPIC scope
definition. (please let me know if anyone knows how I can avoid
user using PCI bus number 0xff... TIA)
- patch 11
- fix comments [Marcel]
- all
- remove intr_supported variable [Marcel]
This patchset provide very basic functionalities for interrupt
remapping (IR) support of the emulated Intel IOMMU device.
By default, IR is disabled to be better compatible with current
QEMU. To enable IR, we can using the following command to boot a
IR-supported VM with virtio-net device with vhost (still do not
support kvm-ioapic, so we need to specify kernel-irqchip={split|off}
here):
$ qemu-system-x86_64 -M q35,iommu=on,intr=on,kernel-irqchip=split \
-enable-kvm -m 1024 \
-netdev tap,id=net0,vhost=on \
-device virtio-net-pci,netdev=user.0 \
-monitor telnet::3333,server,nowait \
/var/lib/libvirt/images/vm1.qcow2
When guest boots, we can verify whether IR enabled by grepping the
dmesg like:
Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: IOAPIC id 0 under DRHD
base 0xfed90000 IOMMU 0
Feb 19 11:21:23 localhost.localdomain kernel: DMAR-IR: Enabled IRQ remapping in
x2apic mode
Currently supported:
- Emulated/Splitted irqchip
- Generic PCI Devices
- vhost devices
- pass through device support? Not tested, but suppose it should work.
- IEC (Interrupt Entry Cache) cache invalidation notification
- EIM (from Jan)
- IRTE Source-id validation
TODO List:
- explicit IEC invalidation (currently, we do update without
checking. Also, we can process QI invalidation in bulk, as Jan
suggested)
- IR fault reporting
- migration support (for IOMMU as general?)
- more?
Jan Kiszka (1):
intel_iommu: Add support for Extended Interrupt Mode
Peter Xu (25):
acpi: enable INTR for DMAR report structure
intel_iommu: allow queued invalidation for IR
intel_iommu: set IR bit for ECAP register
acpi: add DMAR scope definition for root IOAPIC
intel_iommu: define interrupt remap table addr register
intel_iommu: handle interrupt remap enable
intel_iommu: define several structs for IOMMU IR
intel_iommu: provide helper function vtd_get_iommu
intel_iommu: add IR translation faults defines
intel_iommu: Add support for PCI MSI remap
q35: ioapic: add support for emulated IOAPIC IR
ioapic: introduce ioapic_entry_parse() helper
intel_iommu: add support for split irqchip
q35: add "intremap" parameter to enable IR
intel_iommu: introduce IEC notifiers
ioapic: register VT-d IEC invalidate notifier
ioapic: keep RO bits for IOAPIC entry
ioapic: clear remote irr bit for edge-triggered interrupts
intel_iommu: add SID validation for IR
x86-iommu: introduce parent class
x86-iommu: replace existing VT-d hooks into X86 ones
kvm-irqchip: simplify kvm_irqchip_add_msi_route
kvm-irqchip: i386: add hook for add/remove virq
kvm-irqchip: x86: add msi route notify fn
kvm-irqchip: do explicit commit when update irq
hw/core/machine.c | 22 ++
hw/i386/Makefile.objs | 2 +-
hw/i386/acpi-build.c | 38 ++--
hw/i386/intel_iommu.c | 429 +++++++++++++++++++++++++++++++++++++-
hw/i386/intel_iommu_internal.h | 49 ++++-
hw/i386/kvm/pci-assign.c | 10 +-
hw/i386/pc.c | 3 +
hw/i386/x86-iommu.c | 91 ++++++++
hw/intc/ioapic.c | 158 ++++++++++----
hw/misc/ivshmem.c | 4 +-
hw/pci-host/q35.c | 17 +-
hw/pci/pci.c | 15 ++
hw/vfio/pci.c | 12 +-
hw/virtio/virtio-pci.c | 10 +-
include/hw/acpi/acpi-defs.h | 15 ++
include/hw/boards.h | 1 +
include/hw/i386/apic-msidef.h | 1 +
include/hw/i386/intel_iommu.h | 147 ++++++++++++-
include/hw/i386/ioapic_internal.h | 6 +
include/hw/i386/pc.h | 4 +
include/hw/i386/x86-iommu.h | 110 ++++++++++
include/hw/pci-host/q35.h | 9 +
include/hw/pci/pci.h | 2 +
include/sysemu/kvm.h | 21 +-
kvm-all.c | 14 +-
kvm-stub.c | 6 +-
target-arm/kvm.c | 11 +
target-i386/kvm.c | 98 ++++++++-
target-mips/kvm.c | 11 +
target-ppc/kvm.c | 11 +
target-s390x/kvm.c | 11 +
trace-events | 6 +
32 files changed, 1232 insertions(+), 112 deletions(-)
create mode 100644 hw/i386/x86-iommu.c
create mode 100644 include/hw/i386/x86-iommu.h
--
2.4.11
- [Qemu-devel] [PATCH v6 00/26] IOMMU: Enable interrupt remapping for Intel IOMMU,
Peter Xu <=
- [Qemu-devel] [PATCH v6 01/26] acpi: enable INTR for DMAR report structure, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 02/26] intel_iommu: allow queued invalidation for IR, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 03/26] intel_iommu: set IR bit for ECAP register, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 04/26] acpi: add DMAR scope definition for root IOAPIC, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 05/26] intel_iommu: define interrupt remap table addr register, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 06/26] intel_iommu: handle interrupt remap enable, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 07/26] intel_iommu: define several structs for IOMMU IR, Peter Xu, 2016/05/04
- [Qemu-devel] [PATCH v6 08/26] intel_iommu: provide helper function vtd_get_iommu, Peter Xu, 2016/05/04