Re: [Qemu-devel] [PATCH v9 05/11] vfio: add check host bus reset is support or not

[Alex Williamson]

On Thu, 22 Sep 2016 16:34:32 +0800
Dou Liyang <address@hidden> wrote:

> Hi Alex,
> 
> At 09/01/2016 03:56 AM, Alex Williamson wrote:
> > On Tue, 19 Jul 2016 15:38:23 +0800
> > Zhou Jie <address@hidden> wrote:
> >  
> >> From: Chen Fan <address@hidden>
> >>
> >> When assigning a vfio device with AER enabled, we must check whether
> >> the device supports a host bus reset (ie. hot reset) as this may be
> >> used by the guest OS in order to recover the device from an AER
> >> error.  QEMU must therefore have the ability to perform a physical
> >> host bus reset using the existing vfio APIs in response to a virtual
> >> bus reset in the VM.  A physical bus reset affects all of the devices
> >> on the host bus, therefore we place a few simplifying configuration
> >> restriction on the VM:
> >>
> >>  - All physical devices affected by a bus reset must be assigned to
> >>    the VM with AER enabled on each and be configured on the same
> >>    virtual bus in the VM.
> >>
> >>  - No devices unaffected by the bus reset, be they physical, emulated,
> >>    or paravirtual may be configured on the same virtual bus as a
> >>    device supporting AER signaling through vfio.
> >>
> >> In other words users wishing to enable AER on a multifunction device
> >> need to assign all functions of the device to the same virtual bus
> >> and enable AER support for each device.  The easiest way to
> >> accomplish this is to identity map the physical functions to virtual
> >> functions with multifunction enabled on the virtual device.  
> >
> > Why am I able to start the following VM with aer=on for the vfio-pci
> > devices?  
> 
> In my opinion, because, when we check out this situation in vfio_init,
> our code just do nothing except going to out_teardown. so the devices 
> which are behind a PCIe-to-PCI bridge will be regarded as the PCI devices.
> 
> >
> > # lspci -tv
> > -[0000:00]-+-00.0  Intel Corporation 82G33/G31/P35/P31 Express DRAM 
> > Controller
> >            +-01.0  Device 1234:1111
> >            +-1c.0-[01]--
> >            +-1d.0-[02]--+-01.0  Intel Corporation 82576 Gigabit Network 
> > Connection
> >            |            \-01.1  Intel Corporation 82576 Gigabit Network 
> > Connection
> >            ...
> >
> > # lspci -vvv -s 1d.0
> > 00:1d.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge (prog-if 00 [Normal 
> > decode])
> >
> > The devices are behind a PCIe-to-PCI bridge, so shouldn't specifying
> > aer=on for the vfio-pci devices cause a configuration error?  
> 
> Yes, I think it may makes users confuse.
> 
> I added a error_report for it.

I think we need more than an error_report, our strategy to date has
been that if the user specifies aer=on and we cannot support AER on the
device then we fail the initialization of the device.  Eric's patches
to convert vfio to realize and overhaul the error handling might make
this easier to accomplish.  Thanks,

Alex

> 
> Then, I test many passible combinations, such as:
> 
> pci-bridge
>      |- vfio-pci
> 
> -device pci-bridge,chassis_nr=3,id=pci.3,bus=pcie.0,addr=0x1d \
> -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.3,addr=1 \
> 
> error:
> vfio: Unable to enable AER for device 0000:06:00.0, device does not 
> support AER signaling
> 
> --------------------------------------------------------------------
> 
> pci-bridge
>      |- vfio-pci
>      |- vfio-pci
> 
> -device pci-bridge,chassis_nr=3,id=pci.3,bus=pcie.0,addr=0x1d \
> -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.3,addr=1 \
> -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=pci.3,addr=1.1 \
> 
> error:
> vfio: Unable to enable AER for device 0000:06:00.0, device does not 
> support AER signaling
> 
> --------------------------------------------------------------------
> ioh3420
>    |- pci-bridge
>        |- vfio-pci
>        |- vfio-pci
> 
> -device ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1,chassis=1 \
> -device pci-bridge,chassis_nr=3,id=pci.1,bus=bridge1,addr=0x1d \
> -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=pci.1,addr=1 \
> -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=pci.1,addr=1.1 \
> 
> error:
> vfio: Unable to enable AER for device 0000:06:00.0, device does not 
> support AER signaling
> 
> --------------------------------------------------------------------
> just for test:
> pci-bridge
>    |- ioh3420
>        |- vfio-pci
>        |- vfio-pci
> -device pci-bridge,chassis_nr=3,id=pci.1,bus=pcie.0,addr=0x1d \
> -device ioh3420,bus=pci.1,addr=1c.0,port=1,id=bridge1,chassis=1 \
> -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=bridge1,addr=1 \
> -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=bridge1,addr=1.1 \
> 
> error:
> vfio: Unable to enable AER for device 0000:06:00.0, parent bus does not 
> support AER signaling
> 
> --------------------------------------------------------------------
> ioh3420
>     |- vfio-pci
>     |- vfio-pci
> 
> -device ioh3420,bus=pcie.0,addr=1c.0,port=1,id=bridge1,chassis=1 \
> -device vfio-pci,aer=on,host=06:00.0,id=hostdev0,bus=bridge1,addr=1 \
> -device vfio-pci,aer=on,host=06:00.1,id=hostdev1,bus=bridge1,addr=1.1 \
> 
> OK
> --------------------------------------------------------------------
> ...
> 
> Thanks,
> Dou
> 
> >  
> 
> [...]
> 
> 
> >  
> 
> 
> 
>