Re: [Qemu-devel] [RFC v4 00/27] vSMMUv3/pSMMUv3 2 stage VFIO integration

[Zhangfei Gao]

On Thu, Jul 11, 2019 at 1:55 PM Auger Eric <address@hidden> wrote:
>
> Hi Zhangfei,
>
> On 7/11/19 3:53 AM, Zhangfei Gao wrote:
> > On Mon, May 27, 2019 at 7:44 PM Eric Auger <address@hidden> wrote:
> >>
> >> Up to now vSMMUv3 has not been integrated with VFIO. VFIO
> >> integration requires to program the physical IOMMU consistently
> >> with the guest mappings. However, as opposed to VTD, SMMUv3 has
> >> no "Caching Mode" which allows easy trapping of guest mappings.
> >> This means the vSMMUV3 cannot use the same VFIO integration as VTD.
> >>
> >> However SMMUv3 has 2 translation stages. This was devised with
> >> virtualization use case in mind where stage 1 is "owned" by the
> >> guest whereas the host uses stage 2 for VM isolation.
> >>
> >> This series sets up this nested translation stage. It only works
> >> if there is one physical SMMUv3 used along with QEMU vSMMUv3 (in
> >> other words, it does not work if there is a physical SMMUv2).
> >>
> >> The series uses a new kernel user API [1], still under definition.
> >>
> >> - We force the host to use stage 2 instead of stage 1, when we
> >>   detect a vSMMUV3 is behind a VFIO device. For a VFIO device
> >>   without any virtual IOMMU, we still use stage 1 as many existing
> >>   SMMUs expect this behavior.
> >> - We introduce new IOTLB "config" notifiers, requested to notify
> >>   changes in the config of a given iommu memory region. So now
> >>   we have notifiers for IOTLB changes and config changes.
> >> - vSMMUv3 calls config notifiers when STE (Stream Table Entries)
> >>   are updated by the guest.
> >> - We implement a specific UNMAP notifier that conveys guest
> >>   IOTLB invalidations to the host
> >> - We implement a new MAP notifiers only used for MSI IOVAs so
> >>   that the host can build a nested stage translation for MSI IOVAs
> >> - As the legacy MAP notifier is not called anymore, we must make
> >>   sure stage 2 mappings are set. This is achieved through another
> >>   memory listener.
> >> - Physical SMMUs faults are reported to the guest via en eventfd
> >>   mechanism and reinjected into this latter.
> >>
> >> Note: The first patch is a code cleanup and was sent separately.
> >>
> >> Best Regards
> >>
> >> Eric
> >>
> >> This series can be found at:
> >> https://github.com/eauger/qemu/tree/v4.0.0-2stage-rfcv4
> >>
> >> Compatible with kernel series:
> >> [PATCH v8 00/29] SMMUv3 Nested Stage Setup
> >> (https://lkml.org/lkml/2019/5/26/95)
> >>
> >
> > Have tested vfio mode in qemu on arm64 platform.
> >
> > Tested-by: Zhangfei Gao <address@hidden>
> > qemu: https://github.com/eauger/qemu/tree/v4.0.0-2stage-rfcv4
> > kernel: https://github.com/eauger/linux/tree/v5.2-rc1-2stage-v8
>
> Your testing is really appreciated.
>
> Both kernel and QEMU series will be respinned. I am currently waiting
> for 5.3 kernel window as it will resolve some dependencies on the fault
> reporting APIs. My focus is to get the updated kernel series reviewed
> and tested and then refine the QEMU integration accordingly.
>
Thanks Eric, that's great
Since I found kernel part (drivers/iommu/arm-smmu-v3.c) will be
conflicting with Jean's sva patch.
Especially this one: iommu/smmuv3: Dynamically allocate s1_cfg and s2_cfg

Thanks