Re: [Qemu-devel] [PATCH 2/3] Add a new PCI region type to supports 64 bit ranges

[Gerd Hoffmann]

On 12/29/11 03:56, Kevin O'Connor wrote:
> On Wed, Dec 28, 2011 at 06:26:05PM +1300, Alexey Korolev wrote:
>> This patch adds PCI_REGION_TYPE_PREFMEM_64 region type and modifies types of
>> variables to make it possible to work with 64 bit addresses.
>>
>> Why I've added just one region type PCI_REGION_TYPE_PREFMEM_64 and haven't
>> added PCI_REGION_TYPE_MEM_64? According to PCI architecture
>> specification, the
>> bridges can describe 64bit ranges for prefetchable type of memory
>> only. So it's very
>> unlikely that devices exporting 64bit non-prefetchable BARs. Anyway
>> this code will work
>> with 64bit non-prefetchable BARs unless the PCI device is not behind
>> the secondary bus.
> [...]
>> --- a/src/pciinit.c
>> +++ b/src/pciinit.c
>> @@ -22,6 +22,7 @@ enum pci_region_type {
>>      PCI_REGION_TYPE_IO,
>>      PCI_REGION_TYPE_MEM,
>>      PCI_REGION_TYPE_PREFMEM,
>> +    PCI_REGION_TYPE_PREFMEM_64,
>>      PCI_REGION_TYPE_COUNT,
>>  };
> 
> This doesn't seem right.  A 64bit bar is not a new category - it's
> just a way of representing larger values within the existing
> categories.

We need to take care to not confuse things here.  We have pci bars which
can be 64bit capable.  We (will) have a memory region (aka address
space) above 4G.  We may or may not map 64bit bars there.

> Tracking of 64bit prefmem sections separately from
> regular prefmem sections doesn't make sense, because both need to be
> allocated from the same pool when behind a bridge.

Yea, bridges make the picture a bit more complicated.  A bridge itself
might not be 64bit capable, which makes it impossible to map devices
behind it above 4G.  Likewise with multiple devices where one of them
has a 32bit prefmem bar:  The (single) prefmem memory window must be
below 4G then too.

I think we'll have to keep track for each bar whenever we'll go map it
above or below 4G.  For starters this logic can be simply to only map
64bit bars of root bus devices above 4G, which completely sidesteps the
bridge issues for now.

Once qemu has full pci-e support we probably want refine this logic,
which probably requires an extra pass between pci bar discovery and
placing the bars into the memory regions we have.

cheers,
  Gerd