Re: [Qemu-devel] [RFC] Proposal: PCI/PCIe: inbound BAR0 emulation for PCI controller (Root Complex)

[Bhushan Bharat-R65777]

Just adding Alex Graf and address@hidden

Will add my response in reply. 

> -----Original Message-----
> From: Benjamin Herrenschmidt [mailto:address@hidden
> Sent: Friday, June 08, 2012 4:32 PM
> To: Bhushan Bharat-R65777
> Cc: address@hidden; Wood Scott-B07421; Yoder Stuart-B08248
> Subject: Re: [RFC] Proposal: PCI/PCIe: inbound BAR0 emulation for PCI 
> controller
> (Root Complex)
> 
> On Fri, 2012-06-08 at 09:03 +0000, Bhushan Bharat-R65777 wrote:
> > Hi All,
> >
> > When Freescale PCI controller configured in Root Complex mode then,
> > its configuration header (type 1) have one inbound BAR (BAR0, called
> > as CCSRBAR). And rest of BARs (inbound and outbound) are supported by
> > ATMU registers, which are outside the Type 1 configuration header.
> > This BAR0 of Type 1 configuration header always translate to CCSR
> > space and is of the size of CCSR. This BAR0 (inbound window) is
> > required for MSI interrupts support. With this window, the pci devices
> > can write to the MPIC MSI registers to generate MSI interrupt.
> 
> So you should start with giving an overview, nobody here knows what "ATMU" or
> "CCSR" are, those are device specific acronyms
> 
> As a matter of fact it looks like I misunderstood you on IRC :-) IE. I thought
> your problem was that BAR0 is 'special' as it represents the inbound memory
> window (as it does on some PPC 4xx for example and a whole other collection of
> embedded devices). You seem to indicate that it's in fact MMIO:
> 
> > As far as I know, as of now no emulated PCI controller supports this
> > BAR0 in type 1 configuration header. But probably (I think so) that
> > supporting this is not of big concern, but the point is that this
> > window (BAR0) translate to mmio-regs (CCSR) and not to DDR memory.
> 
> So the 4xx case or the case where your BAR 0 actually represents system memory
> are something that is bothering me but not what you are trying to discuss 
> here.
> 
> IE. Normal BAR operations should work for an MMIO BAR, ie register it normally
> as a PCIe device and devices accessing those addresses will do the right 
> thing.
> 
> From there, AFAIK, the MSI code will simply do stl_le_phys, which I
> -believe- will hit a BAR that does MMIO decoding for those addresses, but I'll
> let people knowing qemu more in depth reply whether that's true or not.
> 
> There's very few devices with MSI support in hw/* so it's hard to test with
> anything other than virtio.
> 
> > So I have couple of concerns here:
> >
> > 1. Whenever PCI device does need DMA then these windows (inbound and
> > outbound ATMUs registers) need to used to translate pci address to
> > system physical address (Sometime we also call this as cpu address
> > space). This will probably be done by : [Qemu-devel] [PATCH 00/12]
> > IOMMU Infrastructure : patch-set ( I am trying to understand these
> > patches :-))
> 
> Yes, that's basically it. The patches allow you to add a set of routines that
> will be used for translating DMA accesses to system memory along with 
> map/unmap
> operations etc...
> 
> Beware that this only works with drivers that use the proper accessors.
> The patch series "fixes" some of them but not everything. I don't know off 
> hand
> (I don't have the code at hand right now) whether the MSI code goes through 
> that
> or not, if it does, you may need to be careful to
> -not- hit the translation layer and pass the accesses on.
> 
> > 2. Hook up this inbound BAR0 in the above patch-set to translate to
> > mmio-regs. As this would be controller specific, a callback will be
> > registered for translation. Now it will be upto the controller
> > specific code on how it translates. As this is needed only for MSI
> > interrupt, maybe, initially we do not do anything initially, till we
> > want MSI emulation in QEMU.
> 
> Well, virtio can make good use of MSI emulation ...
> 
> Cheers,
> Ben.
> 
> > Please provide your comment.
> >
> > Thanks
> > -Bharat
> >
> >
> 
>