Re: [Qemu-devel] vfio in the guest: no available reset mechanism

[Michael S. Tsirkin]

On Wed, Jul 30, 2014 at 10:16:32PM +0800, Le Tan wrote:
> Hi Michael,
> 
> 2014-07-30 21:16 GMT+08:00 Michael S. Tsirkin <address@hidden>:
> > On Wed, Jul 30, 2014 at 08:24:04PM +0800, Le Tan wrote:
> >> Hi,
> >> I am testing vfio in L1 with my VT-d emulation project. I assigned one
> >> of the two AHCI controllers in L1 to L2 via vfio. After I ran the QEMU
> >> in L1, it complains that:
> >> qemu-system-x86_64: vfio: Cannot reset device 0000:00:03.0, no
> >> available reset mechanism.
> >> qemu-system-x86_64: vfio: Cannot reset device 0000:00:03.0, no
> >> available reset mechanism.
> >>
> >> Then L2 paused when the SeaBIOS executed in ahci_controller_setup(). I
> >> look into this and found that:
> >> val = ahci_ctrl_readl(ctrl, HOST_CTL);
> >> ahci_ctrl_writel(ctrl, HOST_CTL, val | HOST_CTL_AHCI_EN);
> >> When the BIOS tried to read the HOST_CTL, it returns 0x80000002, which
> >> bit 2 (Interrupt Enable) is 1. The AHCI manual says that this bit
> >> should be cleared by default. So maybe L1 didn't reset the device
> >> before assigning it to L2?
> >> Then the BIOS tried to write back to HOST_CTL and it was stuck here. :(
> >>
> >> So can anyone give me some advice? About the state of PCI device or
> >> bus-level reset?
> >>
> >> Here is the detail of the environment and the way I did the vfio.
> >> 1. lspci in L1 said:
> >> 00:03.0 SATA controller [0106]: Intel Corporation 82801IR/IO/IH
> >> (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02)
> >> 00:1f.0 ISA bridge [0601]: Intel Corporation 82801IB (ICH9) LPC
> >> Interface Controller [8086:2918] (rev 02)
> >> 00:1f.2 SATA controller [0106]: Intel Corporation 82801IR/IO/IH
> >> (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02)
> >> 00:1f.3 SMBus [0c05]: Intel Corporation 82801I (ICH9 Family) SMBus
> >> Controller [8086:2930] (rev 02)
> >> 2. Unbind 00:03.0 and do vfio:
> >> modprobe -r vfio_iommu_type1
> >> modprobe vfio_iommu_type1 allow_unsafe_interrupts=1
> >> modprobe vfio-pci
> >> echo 0000:00:03.0 > /sys/bus/pci/devices/0000\:00\:03.0/driver/unbind
> >> echo "8086 2922" > /sys/bus/pci/drivers/vfio-pci/new_id
> >> 3. run L2 with "-device vfio-pci,host=00:03.0"
> >>
> >> Any help is appreciated! Thanks very much!
> >>
> >> Regards,
> >> Le
> >
> > Clearly, bus level reset can't work for the root bus :)
> 
> Thanks very much!
> I test the vfio with a second-bus ahci controller and it didn't
> complain about the lack of reset mechanism. :) And the return val of
> HOST_CTL is normal now (the same as emulated ahci controller).
> However, it still paused when the BIOS tried to write to the HOST_CTL.
> Do you have any idea?
> And we should just test vfio and legacy pci-assignment with second bus
> devices, not considering the root-bus devices?
> Thanks again!
> 
> Regards,
> Le

If you want it to work for root bus, you need to implement
a device/function level reset mechanism, e.g. using PM or FLR.

> > --
> > MST