Re: [Qemu-devel] PXB changes for QEMU, and extra root buses for OVMF, round 2

[Laszlo Ersek]

On 06/14/15 12:09, Marcel Apfelbaum wrote:
> On 06/13/2015 04:39 PM, Laszlo Ersek wrote:
>> Following up on this cross-posted message, I will send two patch sets,
>> one for QEMU (to qemu-devel) and another for OVMF (to edk2-devel). With
>> both in place, OVMF supports multiple PCI root buses, and SeaBIOS
>> recognizes boot options that reference devices behind PXBs.
>>
>> * Background.
>>
>> Since the last such "bundled" posting, which was
>>
>>    http://thread.gmane.org/gmane.comp.emulators.qemu/342206
>>    PXB fixes for QEMU, and extra root buses for OVMF
>>
>> with subthreads
>>
>>    [edk2] [PATCH 00/21] OvmfPkg: support extra PCI root buses
>>
>>    [qemu-devel] [PATCH 0/4] PXB tweaks and fixes
>>
>> I have posted no updated OVMF series, and posted two updated QEMU series:
>>
>>    http://thread.gmane.org/gmane.comp.emulators.qemu/343098
>>    [qemu-devel] [PATCH v2 0/4] PXB tweaks and fixes
>>
>>    http://thread.gmane.org/gmane.comp.emulators.qemu/343150
>>    [qemu-devel] [PATCH v3 0/6] PXB tweaks and fixes
>>
>> Version 3 of the QEMU series looked okay-ish for a while. Michael
>> applied the first two of those patches, and (I believe) was planning to
>> investigate the 3rd patch ("hw/pci-bridge: create interrupt-less,
>> hotplug-less bridge for PXB"). Plus, I requested Markus's feedback on
>> some "core" QOM stuff.
>>
>> Other than that, everything seemed to work fine between QEMU and OVMF.
>> In the topmost reference I had named a boot option matching problem (as
>> point (7)) and I was writing code for that in the then-upcoming OVMF
>> patches (planned for OVMF v2). In parallel Marcel turned his attention
>> to SeaBIOS, in order to address the same boot order issue in it. Marcel
>> posted:
>>
>>    http://thread.gmane.org/gmane.comp.emulators.qemu/343284
>>    [SeaBIOS] [PATCH] pci: fixes to allow booting from extra root pci
>>                      buses.
>>
>>    http://thread.gmane.org/gmane.comp.emulators.qemu/343298
>>    [SeaBIOS] [PATCH V2] pci: fixes to allow booting from extra root pci
>>                         buses.
>>
>> In the resultant discussion with Kevin, it became clear that minimally
>> some special casing for QEMU would be necessary in the SeaBIOS patch
>> (because SeaBIOS was already handling extra PCI root buses, and
>> differently from how we had expected). Sticking with the existing code
>> was preferred however.
>>
>> The main issue was that SeaBIOS expected the *serial numbers* of the
>> extra PCI root buses in the boot order OFW devpaths, while QEMU was
>> offering the *bus numbers* themselves. Although the bus numbers in
>> question are permanent (and not guest-assigned) on QEMU, this wasn't a
>> good match for SeaBIOS: on physical hardware, Coreboot assigns the extra
>> root bus numbers (before launching SeaBIOS), and SeaBIOS considers only
>> the serial numbers stable (as long as the hardware config itself is
>> stable).
>>
>>   Here's an example:
>>
>>    serial number      bus number
>>    of extra root bus  of extra root bus
>>    SeaBIOS :)         SeaBIOS :(
>>    -----------------  -----------------
>>                  0x1                0x7
>>                  0x2                0xB
>>                  0x3                0xF
>>
>> The serial numbers always start at 1, are consecutive, and they order
>> the extra root buses based on the bus numbers, increasingly.
>>
>> * New stuff.
>>
>> The two series I'm about to post (QEMU v4 and OVMF v2), supersede *all*
>> of the above. No changes are necessary for SeaBIOS. QEMU applies a
>> transformation from the right column to the left column when formatting
>> the OFW nodes. SeaBIOS consumes those nodes directly, while OVMF applies
>> an inverse transform for boot order matching.
>>
>> (
>>
>> Internally, OVMF sticks with the right column in the UID fields of the
>> ACPI_HID_DEVICE_PATH nodes (also known as "PciRoot(UID)" nodes) that it
>> creates alongside the extra PCI root bridge IO protocols, for the extra
>> root buses.
>>
>> This equality is important because those ACPI_HID_DEVICE_PATH.UID fields
>> must in turn match, by requirement of the UEFI spec, the UIDs placed
>> into the SSDT by QEMU's ACPI generator.
>>
>> Therefore, I had to choose between picking the PciRoot(UID) values from
>> the left column -- which would have necessitated parallel changes to
>> QEMU's ACPI generator -- vs. implementing a reverse transform in OVMF's
>> boot order matching. I chose the latter.
>>
>> )
>>
>> Summary:
>> - no ACPI changes in QEMU
>> - no SeaBIOS changes
>> - re-verified interrupt line assignments between SeaBIOS and OVMF: they
>>    continue to match
>> - retested Windows Server 2012 R2 boot & networking on OVMF (with NIC
>>    behind PXB)
>> - regression-tested the "extra root bus"-less case on OVMF
>> - booted a Fedora CD-ROM on a virtio-scsi HBA on a PXB with SeaBIOS
>>    (with -boot strict=on)
>>
>> Thanks
>> Laszlo
>>
> Tested-by: Marcel Apfelbaum <address@hidden>
> Reviewed-by: Marcel Apfelbaum <address@hidden>
> 
> Tested booting with 1 device on main root bus, and 2 devices behind PXBs.
> Changed the boot-order to test all the combinations and verified that
> PXE is loaded from the right devices.

Awesome, thanks! :)

> Thank you Laszlo for your effort to get this done so quickly.
> Now let's wait for Michael/Markus opinion.

Yes, let's. :)

Cheers
Laszlo