Re: [Qemu-devel] Attaching PCI devices to the PCIe root complex

[Laine Stump]

On 09/25/2013 04:48 AM, Marcel Apfelbaum wrote:
> On Wed, 2013-09-25 at 10:01 +0300, Michael S. Tsirkin wrote:
>> On Tue, Sep 24, 2013 at 06:01:02AM -0400, Laine Stump wrote:
>>> When I added support for the Q35-based machinetypes to libvirt, I
>>> specifically prohibited attaching any PCI devices (with the exception of
>>> graphics controllers) to the PCIe root complex,
>> That's wrong I think. Anything attached to RC is an integrated
>> endpoint, and these can be PCI devices.
> I couldn't find on PCIe spec any mention that "Root Complex Integrated 
> EndPoint"
> must be PCIe. But, from spec 1.3.2.3:
> - A Root Complex Integrated Endpoint must not require I/O resources claimed 
> through BAR(s).
> - A Root Complex Integrated Endpoint must not generate I/O Requests.
> - A Root Complex Integrated Endpoint is required to support MSI or MSI-X or 
> both if an
> interrupt resource is requested.

So much easier in the real world, where the rule is "if it fits in the
socket and the pins match up, then it's okay" :-)


>> IMO, we really need to grow an interface to query this kind of thing.
> Basically libvirt needs to know:
> 1. for (libvirt) controllers: what kind of devices can be plugged in

The controllers also need a flag indicating if they supporting having
devices hot-plugged into them. For example, as far as I understand, the
PCI root complex ("pcie-root" in libvirt) doesn't support hot-plugging
devices, nor does i82801b11-bridge ("dmi-to-pci-bridge" in libvirt), but
pci-bridge, ioh3420 ("root-port" in libvirt), and xio3130-downstream
("downstream-switch-port" in libvirt) *do* support hot-plugging devices
(as far as I know, none of these controllers can themselves be
hot-plugged into another controller, but that could change in the
future, e.g. I recall someone saying something about  hot-plug of a
pci-bridge being a future possibility)


> 2. for devices (controller is also a device)
>     - to which controllers can it be plugged in
>     - does it support hot-plug?
> 3. implicit controllers of the machine types (q35 - "pcie-root", i440fx - 
> "pci-root")
> All the above must be exported to libvirt
>
> Implementation options:
> 1. Add a compliance field on PCI/PCIe devices and controllers stating if it 
> supports
>    PCI/PCIe or both (and maybe hot-plug)
>    - consider plug type + compliance to figure out whether a plug can go into 
> a socket
>    
> 2. Use Markus Armbruster idea of introducing a concept of "plug and sockets":
>    - dividing the devices into adapters and plugs
>    - adding sockets to bridges(buses?).
>    In this way it would be clear which devices can connect to bridges

However it is done, each controller will need to have two sets of flags
- one for what it can plug into, and one for what can be plugged into it.