Re: [Qemu-devel] [Question] Why doesn't PCIe hotplug work for Q35 machine?

[Paolo Bonzini]

Il 18/08/2014 11:12, Gonglei (Arei) ha scritto:
> Hi,
> 
>> Subject: Re: [Question] Why doesn't PCIe hotplug work for Q35 machine?
>>
>> On Sun, 2014-08-17 at 13:00 +0200, Michael S. Tsirkin wrote:
>>> On Fri, Aug 15, 2014 at 07:33:29AM +0000, Gonglei (Arei) wrote:
>>>> Hi,
>>>>
>>>> I noticed that the qemu-2.1 release change log says
>>>> " PCIe: Basic hot-plug/hot-unplug support for Q35 machine."
>>>> And then I made a testing for the hotplugging function of Q35.
>>>> But I'm failed, and I got the dmesg log in guest os as below:
>>>>
>>>> [ 159.035250] Pciehp 0000:05:00.0:pcie24: Button pressed on Slot (0 - 4)
>>>> [ 159.035274] Pciehp 0000:05:00.0:pcie24: Card present on Slot (0 - 4)
>>>> [ 159.036517] Pciehp 0000:05:00.0:pcie24: PCI slot #0 - 4 - powering on due
>> to button press.
>>>> [ 159.188049] Pciehp 0000:05:00.0:pcie24: Failed to check link status
>>>> [ 159.201968] Pciehp 0000:05:00.0:pcie24: Card not present on Slot (0 - 4)
>>>> [ 159.202529] Pciehp 0000:05:00.0:pcie24: Already disabled on Slot (0 - 4)
>>>>
>>>> Steps of testing:
>>>>
>>>> #1. QEMU version:
>>>>
>>>>    The lateset master tree source.
>>>>
>>>> #2. Command line:
>>>>
>>>> ./qemu-system-x86_64 -enable-kvm -m 2048 -machine q35 -device
>> ide-drive,bus=ide.2,drive=MacHDD \
>>>>  -drive id=MacHDD,if=none,file=/mnt/sdb/gonglei/image/redhat_q35.img
>> -monitor stdio -vnc :10 -readconfig ../docs/q35-chipset.cfg
>>>> QEMU 2.0.93 monitor - type 'help' for more information
>>>> (qemu) device_add
>> virtio-net-pci,id=nic2,bus=pcie-switch-downstream-port-1-1,addr=1.0
>>>
>>> I don't think you can use any slot except slot 0 for pci express.
> 
> OK. Does the PCIe specification say that?
> I appreciate very much that you explain more.

The closest I could find is in "7.3. Configuration Transaction
Rules"/"7.3.1. Device Number":

With non-ARI Devices, PCI Express components are restricted to
implementing a single Device Number on their primary interface (Upstream
Port) [...] Downstream Ports that do not have ARI Forwarding enabled
must associate only Device 0 with the device attached to the Logical Bus
representing the Link from the Port. Configuration Requests
targeting the Bus Number associated with a Link specifying Device Number
0 are delivered to the device attached to the Link; Configuration
Requests specifying all other Device Numbers (1-31)
must be terminated by the Switch Downstream Port or the Root Port with
an Unsupported Request Completion Status (equivalent to Master Abort in
PCI).

Paolo

>> Indeed, only slot 0 can be used, try the same without specifying the addr
>> property.
>>
> 
> OK, got it. Thank you so much.
> 
>>>> (qemu) info network
>>>> hub 0
>>>>  \ user.0: index=0,type=user,net=10.0.2.0,restrict=off
>>>>  \ e1000.0: index=0,type=nic,model=e1000,macaddr=52:54:00:12:34:56
>>>>
>>>> #3. Guest OS:
>>>>
>>>>    Both rhel-server-7.0-x86_64 and SLES-11-SP3-DVD-x86_64-GM.
>>>>
>>>> #4. Content of ../docs/q35-chipset.cfg:
>>>>
>>>> [snip]
>>>>
>>>> ##
>>>> # Example PCIe switch with two downstream ports
>>>> #
>>>> [device "pcie-switch-upstream-port-1"]
>>>>   driver = "x3130-upstream"
>>>>   bus = "ich9-pcie-port-4"
>>>>   addr = "00.0"
>>>>
>>>> [device "pcie-switch-downstream-port-1-1"]
>>>>   driver = "xio3130-downstream"
>>>>   multifunction = "on"
>>>>   bus = "pcie-switch-upstream-port-1"
>>>>   addr = "00.0"
>>>>   port = "1"
>>>>   chassis = "5"
>>>>
>>>> [device "pcie-switch-downstream-port-1-2"]
>>>>   driver = "xio3130-downstream"
>>>>   multifunction = "on"
>>>>   bus = "pcie-switch-upstream-port-1"
>>>>   addr = "00.1"
>>>>   port = "1"
>>>>   chassis = "6"
>>>>
>>>> [device "ich9-ehci-1"]
>>>>   driver = "ich9-usb-ehci1"
>>>>   multifunction = "on"
>>>>   bus = "pcie.0"
>>>>   addr = "1d.7"
>>>>
>>>> [snip]
>>>>
>>>> BTW, except testing hotplug of downstream ports, I also test the root port
>>>> with "device_add virtio-net-pci,id=nic2,bus=ich9-pcie-port-1,addr=1.0", but
>>>> I got the same result.
>>>
>>> Same thing.
>> Agreed, and also make sure that ich9-pcie-port-1 is a root port's bus and not
>> an upstream port's bus.
>>
> Yes, it is.
> 
>> Thanks,
>> Marcel
>>
>>>
>>>> Am I wrong in some places? Or a QEMU bug?  Thanks for any help.
>>>>
>>>> Best regards,
>>>> -Gonglei
>>>>
>>>
>>> We might want to add a check for this, to produce a
>>> better error message.
>>>
>>
> Work in progress. Thanks!
> 
> I also find other questions. Will post in a patch series.
> 
> Best regards,
> -Gonglei
>