[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-arm] [RFC v3 00/27] vSMMUv3/pSMMUv3 2 stage VFIO integration
From: |
Eric Auger |
Subject: |
[Qemu-arm] [RFC v3 00/27] vSMMUv3/pSMMUv3 2 stage VFIO integration |
Date: |
Fri, 12 Apr 2019 12:03:27 +0200 |
Up to now vSMMUv3 has not been integrated with VFIO. VFIO
integration requires to program the physical IOMMU consistently
with the guest mappings. However, as opposed to VTD, SMMUv3 has
no "Caching Mode" which allows easy trapping of guest mappings.
This means the vSMMUV3 cannot use the same VFIO integration as VTD.
However SMMUv3 has 2 translation stages. This was devised with
virtualization use case in mind where stage 1 is "owned" by the
guest whereas the host uses stage 2 for VM isolation.
This series sets up this nested translation stage. It only works
if there is one physical SMMUv3 used along with QEMU vSMMUv3 (in
other words, it does not work if there is a physical SMMUv2).
The series uses a new kernel user API [1], still under definition.
- We force the host to use stage 2 instead of stage 1, when we
detect a vSMMUV3 is behind a VFIO device. For a VFIO device
without any virtual IOMMU, we still use stage 1 as many existing
SMMUs expect this behavior.
- We introduce new IOTLB "config" notifiers, requested to notify
changes in the config of a given iommu memory region. So now
we have notifiers for IOTLB changes and config changes.
- vSMMUv3 calls config notifiers when STE (Stream Table Entries)
are updated by the guest.
- We implement a specific UNMAP notifier that conveys guest
IOTLB invalidations to the host
- We implement a new MAP notifiers only used for MSI IOVAs so
that the host can build a nested stage translation for MSI IOVAs
- As the legacy MAP notifier is not called anymore, we must make
sure stage 2 mappings are set. This is achieved through another
memory listener.
- Physical SMMUs faults are reported to the guest via en eventfd
mechanism and reinjected into this latter.
Note: The 2 first patches were sent separately.
Best Regards
Eric
This series can be found at:
https://github.com/eauger/qemu/tree/v4.0.0-rc2-2stage-rfcv3
Compatible with kernel series:
[PATCH v7 00/23] SMMUv3 Nested Stage Setup
https://github.com/eauger/linux/tree/v5.1-rc3-2stage-v7
History:
v2 -> v3:
- rework fault handling
- MSI binding registration done in vfio-pci. MSI binding tear down called
on container cleanup path
- leaf parameter propagated
v1 -> v2:
- Fixed dual assignment (asid now correctly propagated on TLB invalidations)
- Integrated fault reporting
Next Steps:
- Mature the user API with people involved in SVA work (KVM forum may be a
good opportunity to meet)
- Submit the IOMMU cfg notifier changes and some VFIO changes separately, to
progress independently on the kernel dependency
Eric Auger (27):
hw/arm/smmuv3: Remove SMMUNotifierNode
vfio/common: Introduce vfio_set_irq_signaling helper
update-linux-headers: Import iommu.h
header update against 5.1-rc3 and IOMMU/VFIO nested stage APIs
memory: add IOMMU_ATTR_VFIO_NESTED IOMMU memory region attribute
memory: add IOMMU_ATTR_MSI_TRANSLATE IOMMU memory region attribute
hw/arm/smmuv3: Advertise VFIO_NESTED and MSI_TRANSLATE attributes
hw/vfio/common: Force nested if iommu requires it
memory: Prepare for different kinds of IOMMU MR notifiers
memory: Add IOMMUConfigNotifier
memory: Add arch_id and leaf fields in IOTLBEntry
hw/arm/smmuv3: Store the PASID table GPA in the translation config
hw/arm/smmuv3: Implement dummy replay
hw/arm/smmuv3: Fill the IOTLBEntry arch_id on NH_VA invalidation
hw/arm/smmuv3: Fill the IOTLBEntry leaf field on NH_VA invalidation
hw/arm/smmuv3: Notify on config changes
hw/vfio/common: Introduce vfio_alloc_guest_iommu helper
hw/vfio/common: Introduce hostwin_from_range helper
hw/vfio/common: Introduce helpers to DMA map/unap a RAM section
hw/vfio/common: Setup nested stage mappings
hw/vfio/common: Register a MAP notifier for MSI binding
vfio-pci: Expose MSI stage 1 bindings to the host
memory: Introduce IOMMU Memory Region inject_faults API
hw/arm/smmuv3: Implement fault injection
vfio-pci: register handler for iommu fault
vfio-pci: Set up fault regions
vfio-pci: Implement the DMA fault handler
exec.c | 12 +-
hw/arm/smmu-common.c | 16 +-
hw/arm/smmuv3.c | 212 +++++--
hw/arm/trace-events | 3 +-
hw/i386/amd_iommu.c | 2 +-
hw/i386/intel_iommu.c | 25 +-
hw/misc/tz-mpc.c | 8 +-
hw/ppc/spapr_iommu.c | 2 +-
hw/s390x/s390-pci-inst.c | 4 +-
hw/vfio/common.c | 548 ++++++++++++++----
hw/vfio/pci.c | 469 +++++++++------
hw/vfio/pci.h | 4 +
hw/vfio/platform.c | 55 +-
hw/vfio/trace-events | 6 +-
hw/virtio/vhost.c | 14 +-
include/exec/memory.h | 159 ++++-
include/hw/arm/smmu-common.h | 9 +-
include/hw/vfio/vfio-common.h | 10 +
.../infiniband/hw/vmw_pvrdma/pvrdma_dev_api.h | 15 +-
include/standard-headers/drm/drm_fourcc.h | 63 ++
include/standard-headers/linux/ethtool.h | 33 +-
.../linux/input-event-codes.h | 2 +
include/standard-headers/linux/input.h | 6 +-
include/standard-headers/linux/pci_regs.h | 2 +
.../standard-headers/linux/virtio_config.h | 6 +
include/standard-headers/linux/virtio_ring.h | 10 -
.../standard-headers/rdma/vmw_pvrdma-abi.h | 1 +
linux-headers/asm-arm/unistd-common.h | 22 +
linux-headers/asm-arm64/unistd.h | 2 +
linux-headers/asm-generic/unistd.h | 158 +++--
linux-headers/asm-mips/unistd_n32.h | 20 +
linux-headers/asm-mips/unistd_o32.h | 30 +
linux-headers/asm-powerpc/kvm.h | 2 +
linux-headers/asm-powerpc/unistd_32.h | 30 +
linux-headers/asm-powerpc/unistd_64.h | 11 +
linux-headers/asm-s390/unistd_32.h | 33 ++
linux-headers/asm-s390/unistd_64.h | 14 +
linux-headers/asm-x86/unistd_32.h | 34 ++
linux-headers/asm-x86/unistd_64.h | 4 +
linux-headers/asm-x86/unistd_x32.h | 4 +
linux-headers/linux/iommu.h | 240 ++++++++
linux-headers/linux/vfio.h | 107 ++++
memory.c | 69 ++-
scripts/update-linux-headers.sh | 2 +-
44 files changed, 1965 insertions(+), 513 deletions(-)
create mode 100644 linux-headers/linux/iommu.h
--
2.20.1
- [Qemu-arm] [RFC v3 00/27] vSMMUv3/pSMMUv3 2 stage VFIO integration,
Eric Auger <=
- [Qemu-arm] [RFC v3 01/27] hw/arm/smmuv3: Remove SMMUNotifierNode, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 02/27] vfio/common: Introduce vfio_set_irq_signaling helper, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 04/27] header update against 5.1-rc3 and IOMMU/VFIO nested stage APIs, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 03/27] update-linux-headers: Import iommu.h, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 05/27] memory: add IOMMU_ATTR_VFIO_NESTED IOMMU memory region attribute, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 06/27] memory: add IOMMU_ATTR_MSI_TRANSLATE IOMMU memory region attribute, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 07/27] hw/arm/smmuv3: Advertise VFIO_NESTED and MSI_TRANSLATE attributes, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 08/27] hw/vfio/common: Force nested if iommu requires it, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 09/27] memory: Prepare for different kinds of IOMMU MR notifiers, Eric Auger, 2019/04/12
- [Qemu-arm] [RFC v3 10/27] memory: Add IOMMUConfigNotifier, Eric Auger, 2019/04/12