Re: [Qemu-devel] [PATCH RFC] docs: add PCIe devices placement guidelines

[Laszlo Ersek]

On 09/06/16 13:35, Gerd Hoffmann wrote:
>   Hi,
> 
>>> +Plug only legacy PCI devices as Root Complex Integrated Devices
>>> +even if the PCIe spec does not forbid PCIe devices.
>>
>> I suggest "even though the PCI Express spec does not forbid PCI Express
>> devices as Integrated Devices". (Detail is good!)
> 
> While talking about integrated devices:  There is docs/q35-chipset.cfg,
> which documents how to mimic q35 with integrated devices as close and
> complete as possible.
> 
> Usage:
>   qemu-system-x86_64 -M q35 -readconfig docs/q35-chipset.cfg $args
> 
> Side note for usb: In practice you don't want to use the tons of
> uhci/ehci controllers present in the original q35 but plug xhci into one
> of the pcie root ports instead (unless your guest doesn't support xhci).
> 
>>> +as required by PCI spec will reserve a 4K IO range for each.
>>> +The firmware used by QEMU (SeaBIOS/OVMF) will further optimize
>>> +it by allocation the IO space only if there is at least a device
>>> +with IO BARs plugged into the bridge.
>>
>> This used to be true, but is no longer true, for OVMF. And I think it's
>> actually correct: we *should* keep the 4K IO reservation per PCI-PCI bridge.
>>
>> (But, certainly no IO reservation for PCI Express root port, upstream
>> port, or downstream port! And i'll need your help for telling these
>> apart in OVMF.)
> 
> IIRC the same is true for seabios, it looks for the pcie capability and
> skips io space allocation on pcie ports only.
> 
> Side note: the linux kernel allocates io space nevertheless, so
> checking /proc/ioports after boot doesn't tell you what the firmware
> did.

Yeah, we've got to convince Linux to stop doing that. Earlier Alex
mentioned the "hpiosize" and "hpmemsize" PCI subsystem options for the
kernel:

          hpiosize=nn[KMG]        The fixed amount of bus space which is
                          reserved for hotplug bridge's IO window.
                          Default size is 256 bytes.
          hpmemsize=nn[KMG]       The fixed amount of bus space which is
                          reserved for hotplug bridge's memory window.
                          Default size is 2 megabytes.

This document (once complete) would be the basis for tweaking that stuff
in the kernel too. Primarily, "hpiosize" should default to zero, because
its current nonzero default (which gets rounded up to 4KB somewhere) is
what exhausts the IO space, if we have more than a handful of PCI
Express downstream / root ports.

Maybe we can add a PCI quirk for this to the kernel, for QEMU's PCI
Express ports (all of them -- root, upstream, downstream).

Thanks
Laszlo