Re: [Qemu-devel] [PATCH 0/2] intel_iommu: Add support for translation for devices behind bridges

[Michael S. Tsirkin]

On Tue, Oct 21, 2014 at 09:07:40AM -0600, Alex Williamson wrote:
> On Tue, 2014-10-21 at 13:15 +0200, Alexander Graf wrote:
> > 
> > On 21.10.14 11:35, Knut Omang wrote:
> > > On Tue, 2014-10-21 at 11:07 +0200, Alexander Graf wrote:
> > >>
> > >>
> > >>> Am 21.10.2014 um 07:26 schrieb Knut Omang <address@hidden>:
> > >>>
> > >>>> On Tue, 2014-10-21 at 01:29 +0200, Alexander Graf wrote:
> > >>>>
> > >>>>
> > >>>>> Am 21.10.2014 um 00:34 schrieb Knut Omang <address@hidden>:
> > >>>>>
> > >>>>> This patch set changes the data structure used to handle address 
> > >>>>> spaces within
> > >>>>> the emulated Intel iommu to support traversal also if bus numbers are 
> > >>>>> dynamically
> > >>>>> allocated, as is the case for devices that sit behind root ports or 
> > >>>>> downstream switches.
> > >>>>> This means that we cannot use bus number as index, instead a QLIST is 
> > >>>>> used.
> > >>>>>
> > >>>>> This requires a change in the API for setup of IOMMUs which is taken 
> > >>>>> care of by 
> > >>>>> the first patch. The second patch implements the fix.
> > >>>>
> > >>>> Are you sure that this works on real hardware? How does that one
> > >>>> communicate sub-bridge liodns to the iommu? How do they get indexed
> > >>>> from software?
> > >>>
> > >>> I do not claim to fully understand the details of how this is
> > >>> implemented in hardware, but I believe the implementation I propose here
> > >>> should be functionally equivalent to what the Intel IOMMU offers, and
> > >>> similar to the original implementation here, except that the data
> > >>> structure is valid also before enumeration when behind buses.
> > >>
> > >> Can you please give me a pointer to the vt-d spec's section that 
> > >> explains iommu behavior behind bridges?
> > >>
> > >> I've also added Alex W who has played with PCI bridges behind iommus 
> > >> quite a bit recently.
> > >>
> > >>>
> > >>> After enumeration, the only difference would be that during
> > >>> invalidation, there is a list search for the right bus rather than an
> > >>> index lookup as before, slightly less efficient but at the benefit of
> > >>> being independent of bus numbering during setup.
> > >>
> > >> I don't think the implementation is bad, I'm just not sure that it 
> > >> follows the spec, 
> > >> so I want to confirm :).
> > > 
> > > http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf
> > 
> > So if I understand that document correctly, a PCIe / PCI-X bridge can
> > swizzle the requester id depending on a device behind itself. PCI
> > bridges can not - there everything behind the bridge will appear as if
> > the DMA originated from the bridge device.
> > 
> > So conceptually, PCIe / PCI-X bridges should probably be the ones
> > converting requester IDs.
> 
> PCIe-to-PCI/X bridges alias requester IDs to the subordinate bus, so all
> requests appear to come from subordinate-bus-number:00.0.  PCI-X does
> support a requester ID, but there are numerous rules where the bridge
> can take ownership of the transaction that require the IOMMU to handle
> bridges as an alias of the device.  PCI bridges alias all downstream
> devices as the bridge itself, but if you look at
> quirk_use_pcie_bridge_dma_alias() in the kernel, there are numerous
> cases where bridges behave like a PCIe-to-PCI bridge, but fail to
> include a PCIe capability.  Thanks,
> 
> Alex

In fact, apprently exactly 4 such bridges exist. We don't emulate them
nor do we emulate pci-x so it's pretty simple I think.

-- 
MST