Re: [Qemu-devel] [PATCH v1 0/3] intel_iommu: support scalable mode

[Yi Sun]

On 19-03-05 12:48:36, Peter Xu wrote:
> On Tue, Mar 05, 2019 at 11:24:24AM +0800, Yi Sun wrote:
> > On 19-03-05 11:09:34, Peter Xu wrote:
> > > On Tue, Mar 05, 2019 at 10:34:52AM +0800, Yi Sun wrote:
> > > > Intel vt-d rev3.0 [1] introduces a new translation mode called
> > > > 'scalable mode', which enables PASID-granular translations for
> > > > first level, second level, nested and pass-through modes. The
> > > > vt-d scalable mode is the key ingredient to enable Scalable I/O
> > > > Virtualization (Scalable IOV) [2] [3], which allows sharing a
> > > > device in minimal possible granularity (ADI - Assignable Device
> > > > Interface). As a result, previous Extended Context (ECS) mode
> > > > is deprecated (no production ever implements ECS).
> > > > 
> > > > This patch set emulates a minimal capability set of VT-d scalable
> > > > mode, equivalent to what is available in VT-d legacy mode today:
> > > >     1. Scalable mode root entry, context entry and PASID table
> > > >     2. Seconds level translation under scalable mode
> > > >     3. Queued invalidation (with 256 bits descriptor)
> > > >     4. Pass-through mode
> > > > 
> > > > Corresponding intel-iommu driver support will be included in
> > > > kernel 5.0:
> > > >     https://www.spinics.net/lists/kernel/msg2985279.html
> > > > 
> > > > We will add emulation of full scalable mode capability along with
> > > > guest iommu driver progress later, e.g.:
> > > >     1. First level translation
> > > >     2. Nested translation
> > > >     3. Per-PASID invalidation descriptors
> > > >     4. Page request services for handling recoverable faults
> > > > 
> > > > To verify the patches, below cases were tested according to Peter Xu's
> > > > suggestions.
> > > >     
> > > > +---------+----------------------------------------------------------------+----------------------------------------------------------------+
> > > >     |         |                      w/ Device Passthr                  
> > > >        |                     w/o Device Passthr                         
> > > > |
> > > >     |         
> > > > +-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > > >     |         | virtio-net-pci, vhost=on      | virtio-net-pci, 
> > > > vhost=off      | virtio-net-pci, vhost=on      | virtio-net-pci, 
> > > > vhost=off      |
> > > >     |         
> > > > +-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > > >     |         | netperf | kernel bld | data cp| netperf | kernel bld | 
> > > > data cp | netperf | kernel bld | data cp| netperf | kernel bld | data 
> > > > cp |
> > > >     
> > > > +---------+-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > > >     | Legacy  | Pass    | Pass       | Pass   | Pass    | Pass       | 
> > > > Pass    | Pass    | Pass       | Pass   | Pass    | Pass       | Pass   
> > > >  |
> > > >     
> > > > +---------+-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > > >     | Scalable| Pass    | Pass       | Pass   | Pass    | Pass       | 
> > > > Pass    | Pass    | Pass       | Pass   | Pass    | Pass       | Pass   
> > > >  |
> > > >     
> > > > +---------+-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > > 
> > > Legacy vfio-pci?
> > > 
> > I tested legacy mode with below command.
> > 
> > w/o passthr
> >     -device intel-iommu \
> >     -netdev tap,id=net1,vhost=off|on \
> >     -device virtio-net-pci,netdev=net1 \
> 
> Virtio devices require explicit enabling of DMAR:
> 
> https://wiki.qemu.org/Features/VT-d#With_Virtio_Devices
> 
> Otherwise it could probably still be bypassing the translation unit.
> 
> If you retest any, please feel free to skip some of the tests like
> kernel build.  Quick netperf should work, which can save you time.
> 
Thanks! Just made a quick test. It works well. Please check below
command.

    -device intel-iommu,intremap=on,device-iotlb=on \
    -netdev tap,id=net1,vhost=off \
    -device ioh3420,id=pcie.1,chassis=1 \
    -device
virtio-net-pci,bus=pcie.1,netdev=net1,disable-legacy=on,disable-modern=off,iommu_platform=on,ats=on
\

> > 
> > w/ passthr: tried net card, graphic card, and a MDEV (cmd is different)
> >     -device intel-iommu,caching-mode=on \
> >     -netdev tap,id=net1,vhost=off|on \
> >     -device virtio-net-pci,netdev=net1 \
> >     -device vfio-pci,host=0000:00:1f.6 \
> 
> This one seems correct to me, and yes covering net/graphic cards
> should be far enough.
> 
> Thanks,
> 
> -- 
> Peter Xu