Re: [Qemu-devel] [PATCH v1 0/3] intel_iommu: support scalable mode

[Peter Xu]

On Tue, Mar 05, 2019 at 11:24:24AM +0800, Yi Sun wrote:
> On 19-03-05 11:09:34, Peter Xu wrote:
> > On Tue, Mar 05, 2019 at 10:34:52AM +0800, Yi Sun wrote:
> > > Intel vt-d rev3.0 [1] introduces a new translation mode called
> > > 'scalable mode', which enables PASID-granular translations for
> > > first level, second level, nested and pass-through modes. The
> > > vt-d scalable mode is the key ingredient to enable Scalable I/O
> > > Virtualization (Scalable IOV) [2] [3], which allows sharing a
> > > device in minimal possible granularity (ADI - Assignable Device
> > > Interface). As a result, previous Extended Context (ECS) mode
> > > is deprecated (no production ever implements ECS).
> > > 
> > > This patch set emulates a minimal capability set of VT-d scalable
> > > mode, equivalent to what is available in VT-d legacy mode today:
> > >     1. Scalable mode root entry, context entry and PASID table
> > >     2. Seconds level translation under scalable mode
> > >     3. Queued invalidation (with 256 bits descriptor)
> > >     4. Pass-through mode
> > > 
> > > Corresponding intel-iommu driver support will be included in
> > > kernel 5.0:
> > >     https://www.spinics.net/lists/kernel/msg2985279.html
> > > 
> > > We will add emulation of full scalable mode capability along with
> > > guest iommu driver progress later, e.g.:
> > >     1. First level translation
> > >     2. Nested translation
> > >     3. Per-PASID invalidation descriptors
> > >     4. Page request services for handling recoverable faults
> > > 
> > > To verify the patches, below cases were tested according to Peter Xu's
> > > suggestions.
> > >     
> > > +---------+----------------------------------------------------------------+----------------------------------------------------------------+
> > >     |         |                      w/ Device Passthr                    
> > >      |                     w/o Device Passthr                         |
> > >     |         
> > > +-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > >     |         | virtio-net-pci, vhost=on      | virtio-net-pci, vhost=off 
> > >      | virtio-net-pci, vhost=on      | virtio-net-pci, vhost=off      |
> > >     |         
> > > +-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > >     |         | netperf | kernel bld | data cp| netperf | kernel bld | 
> > > data cp | netperf | kernel bld | data cp| netperf | kernel bld | data cp |
> > >     
> > > +---------+-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > >     | Legacy  | Pass    | Pass       | Pass   | Pass    | Pass       | 
> > > Pass    | Pass    | Pass       | Pass   | Pass    | Pass       | Pass    |
> > >     
> > > +---------+-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > >     | Scalable| Pass    | Pass       | Pass   | Pass    | Pass       | 
> > > Pass    | Pass    | Pass       | Pass   | Pass    | Pass       | Pass    |
> > >     
> > > +---------+-------------------------------+--------------------------------+-------------------------------+--------------------------------+
> > 
> > Legacy vfio-pci?
> > 
> I tested legacy mode with below command.
> 
> w/o passthr
>     -device intel-iommu \
>     -netdev tap,id=net1,vhost=off|on \
>     -device virtio-net-pci,netdev=net1 \

Virtio devices require explicit enabling of DMAR:

https://wiki.qemu.org/Features/VT-d#With_Virtio_Devices

Otherwise it could probably still be bypassing the translation unit.

If you retest any, please feel free to skip some of the tests like
kernel build.  Quick netperf should work, which can save you time.

> 
> w/ passthr: tried net card, graphic card, and a MDEV (cmd is different)
>     -device intel-iommu,caching-mode=on \
>     -netdev tap,id=net1,vhost=off|on \
>     -device virtio-net-pci,netdev=net1 \
>     -device vfio-pci,host=0000:00:1f.6 \

This one seems correct to me, and yes covering net/graphic cards
should be far enough.

Thanks,

-- 
Peter Xu