Re: [Qemu-devel] Expand ECAM region in machvirt 2_13?

[Ard Biesheuvel]

On 2 May 2018 at 13:31, Laszlo Ersek <address@hidden> wrote:
> On 05/01/18 17:59, Auger Eric wrote:
>> Hi,
>>
>> I would like to resume the discussion on extending the number of PCI
>> buses to 256 (as in Q35) as a follow-up of past discussions:
>> https://lists.gnu.org/archive/html/qemu-devel/2018-01/msg03631.html.
>>
>> With the current 16MB ECAM region we are limited to 16 PCIe busses.
>>
>> Could we envision to have a 256MB ECAM region and move it to another
>> location beyond 256GB, in virt2_13 machine type?
>>
>> Current ECAM range within [0x3f000000, 0x40000000]
>> would be kept unchanged for legacy and when vms->highmem is set to false.
>> Migration from <2_13 to >=2_13 would be allowed whereas migration from
>>> =2.13 to <2.13 wouldn't.
>
> If I understand correctly, the idea is to *move* the current one range,
> if the virt machine type is >= 2.13 and highmem is set to true (which is
> the default IIUC, from 2.12 onward).
>
> For 64-bit (AARCH64) ArmVirtQemu, that should work fine. The firmware
> takes the ECAM base and size from the "pci-host-ecam-generic" DT node,
> property "reg", uint64_t elements #0 and #1. (Sorry if this isn't exact
> DT lingo, I'm paraphrasing the firmware source code.) If the QEMU patch
> just changes the values, that should work transparently.
>
> For 32-bit (ARM) ArmVirtQemu, this change (the new ECAM default) could
> be a problem. PCI stuff in the firmware wouldn't work unless people
> specified highmem=off on the QEMU command line.
>
> Now, I notice highmen defaults to "on" starting with 2.12 even for
> "qemu-system-arm -M virt", not just "qemu-system-aarch64 -M virt", so
> why doesn't that already cause a problem with PCI in the 32-bit guest fw?
>
> Because, currently "highmen" only controls the presence of the 64-bit
> PCI MMIO aperture for BAR allocation; it has no effect on config space.
> And if the 64-bit PCI MMIO aperture is exposed to the 32-bit guest
> firmware, the latter simply ignores the former, and works with the
> 32-bit aperture solely (which is always there).
>
> So, for "qemu-system-arm -M virt" compatibility, I think we might need a
> separate machine type property, which should default to "on" only on
> qemu-system-aarch64 (if such distinctions are allowed).
>
> Of course, I can't tell if the 32-bit ArmVirtQemu firmware is possible
> to run on "qemu-system-aarch64 -M virt". (I think it is; I recall
> something something about ARMv8 having ARMv7 compat, but I don't
> remember ever trying.) If that's the case, then even the above
> suggestion won't work, because it would break 32-bit guest fw that the
> user has run (for whatever reason) on "qemu-system-aarch64 -M virt". In
> this case, I believe we can't just change the contents of the current
> "pci-host-ecam-generic" node, but we should implement some structural
> DTB addition that old firmware will simply not notice, while new
> (64-bit) firmware will specifically look for (and prefer over the old DT
> stuff).
>
> Ard, what's your take? (Sorry if you've already followed up, my email
> processing lags.)
>

Do we have any examples of ACPI platforms where the config space is
mapped above 4 GB?
I'd like to make sure that all existing code copes with that before
even considering it.