[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [RESEND PATCH 2/6] memory: introduce AddressSpaceOps an
From: |
David Gibson |
Subject: |
Re: [Qemu-devel] [RESEND PATCH 2/6] memory: introduce AddressSpaceOps and IOMMUObject |
Date: |
Wed, 20 Dec 2017 22:01:10 +1100 |
User-agent: |
Mutt/1.9.1 (2017-09-22) |
On Wed, Dec 20, 2017 at 02:32:42PM +0800, Liu, Yi L wrote:
> On Mon, Dec 18, 2017 at 10:22:18PM +1100, David Gibson wrote:
> > On Mon, Dec 18, 2017 at 05:17:35PM +0800, Liu, Yi L wrote:
> > > On Mon, Dec 18, 2017 at 05:14:42PM +1100, David Gibson wrote:
> > > > On Thu, Nov 16, 2017 at 04:57:09PM +0800, Liu, Yi L wrote:
[snip]
> > > > So for a typical PAPR setup, the device can access system RAM either
> > > > via DMA in the range 0..1GiB (the "32-bit window") or in the range
> > > > 2^59..2^59+<something> (the "64-bit window"). Typically the 32-bit
> > > > window has mappings dynamically created by drivers, and the 64-bit
> > > > window has all of system RAM mapped 1:1, but that's entirely up to the
> > > > OS, it can map each window however it wants.
> > > >
> > > > 32-bit devices (or "64 bit" devices which don't actually implement
> > > > enough the address bits) will only be able to use the 32-bit window,
> > > > of course.
> > > >
> > > > MMIOs of other devices, the "magic" MSI-X addresses belonging to the
> > > > host bridge and other things exist outside those ranges. Those are
> > > > just the ranges which are used to DMA to RAM.
> > > >
> > > > Each PE (domain) can see a different version of what's in each
> > > > window.
> > >
> > > If I'm correct so far. PE actually defines a mapping between an address
> > > range of an address space(aka. iova address space) and an address range
> > > of the system physical address space.
> >
> > No. A PE means several things, but basically it is an isolation
> > domain, like an Intel IOMMU domain. Each PE has an independent set of
> > IOMMU mappings which translate part of the PCI address space to system
> > memory space.
> >
> > > Then my question is: does each PE define a separate iova address sapce
> > > which is flat from 0 - 2^AW -1, AW is address width? As a reference,
> > > VT-d domain defines a flat address space for each domain.
> >
> > Partly. Each PE has an address space which all devices in the PE see.
> > Only some of that address space is mapped to system memory though,
> > other parts are occupied by devices, others are unmapped.
> >
> > Only the parts mapped by the IOMMU vary between PEs - the other parts
> > of the address space will be identical for all PEs on the host
>
> Thx, this comment addressed me well. This is different from what we have
> on VT-d.
Really? That's hard to believe. I'm pretty sure the VT-d IOMMU must
have a range < 2^64, and anything on the bus outside that range I
expect would be common between all domains. In particular I'd expect
the BARs for other devices not to be remapped by the IOMMU (though
they may be inaccessible on PCI-E due peer to peer transactions being
blocked). As well as things above the IOMMU's range, I'd expect the
region for 32-bit BARs to be common between all domains.
[snip]
> > > > > > IIUC how SVM works, the whole point is that the device
> > > > > > no longer writes into a specific PCI address space. Instead, it
> > > > > > writes directly into a process address space. So it seems to me
> > > > > > more
> > > > > > that SVM should operate at the PCI level, and disassociate the
> > > > > > device
> > > > > > from the normal PCI address space entirely, rather than hooking up
> > > > > > something via that address space.
>
> After thinking more, I agree that it is not suitable to hook up something for
> 1st level via the PCI address space. In the time 1st and 2nd level translation
> is exposed to guest, a device would write to multiple address spaces. PCI
> address
> space is only one of them. I think your reply in another email is a good
> start,
> let me reply my thoughts under that email.
>
> Regards,
> Yi L
>
> > > > >
> > > > > As Peter replied, we still need the PCI address space, it would be
> > > > > used
> > > > > to build up the 2nd level page table which would be used in nested
> > > > > translation.
> > > > >
> > > > > Thanks,
> > > > > Yi L
> > > > >
> > > > > >
> > > > >
> > > >
> > >
> > > Regards,
> > > Yi L
> > >
> >
>
>
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
signature.asc
Description: PGP signature
Re: [Qemu-devel] [RESEND PATCH 2/6] memory: introduce AddressSpaceOps and IOMMUObject, David Gibson, 2017/12/18
Re: [Qemu-devel] [RESEND PATCH 2/6] memory: introduce AddressSpaceOps and IOMMUObject, David Gibson, 2017/12/18
Re: [Qemu-devel] [RESEND PATCH 2/6] memory: introduce AddressSpaceOps and IOMMUObject, David Gibson, 2017/12/18