Re: [Qemu-devel] vfio in the guest: no available reset mechanism

[Alex Williamson]

On Wed, 2014-07-30 at 22:16 +0800, Le Tan wrote:
> Hi Michael,
> 
> 2014-07-30 21:16 GMT+08:00 Michael S. Tsirkin <address@hidden>:
> > On Wed, Jul 30, 2014 at 08:24:04PM +0800, Le Tan wrote:
> >> Hi,
> >> I am testing vfio in L1 with my VT-d emulation project. I assigned one
> >> of the two AHCI controllers in L1 to L2 via vfio. After I ran the QEMU
> >> in L1, it complains that:
> >> qemu-system-x86_64: vfio: Cannot reset device 0000:00:03.0, no
> >> available reset mechanism.
> >> qemu-system-x86_64: vfio: Cannot reset device 0000:00:03.0, no
> >> available reset mechanism.
> >>
> >> Then L2 paused when the SeaBIOS executed in ahci_controller_setup(). I
> >> look into this and found that:
> >> val = ahci_ctrl_readl(ctrl, HOST_CTL);
> >> ahci_ctrl_writel(ctrl, HOST_CTL, val | HOST_CTL_AHCI_EN);
> >> When the BIOS tried to read the HOST_CTL, it returns 0x80000002, which
> >> bit 2 (Interrupt Enable) is 1. The AHCI manual says that this bit
> >> should be cleared by default. So maybe L1 didn't reset the device
> >> before assigning it to L2?
> >> Then the BIOS tried to write back to HOST_CTL and it was stuck here. :(
> >>
> >> So can anyone give me some advice? About the state of PCI device or
> >> bus-level reset?
> >>
> >> Here is the detail of the environment and the way I did the vfio.
> >> 1. lspci in L1 said:
> >> 00:03.0 SATA controller [0106]: Intel Corporation 82801IR/IO/IH
> >> (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02)
> >> 00:1f.0 ISA bridge [0601]: Intel Corporation 82801IB (ICH9) LPC
> >> Interface Controller [8086:2918] (rev 02)
> >> 00:1f.2 SATA controller [0106]: Intel Corporation 82801IR/IO/IH
> >> (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode] [8086:2922] (rev 02)
> >> 00:1f.3 SMBus [0c05]: Intel Corporation 82801I (ICH9 Family) SMBus
> >> Controller [8086:2930] (rev 02)
> >> 2. Unbind 00:03.0 and do vfio:
> >> modprobe -r vfio_iommu_type1
> >> modprobe vfio_iommu_type1 allow_unsafe_interrupts=1
> >> modprobe vfio-pci
> >> echo 0000:00:03.0 > /sys/bus/pci/devices/0000\:00\:03.0/driver/unbind
> >> echo "8086 2922" > /sys/bus/pci/drivers/vfio-pci/new_id
> >> 3. run L2 with "-device vfio-pci,host=00:03.0"
> >>
> >> Any help is appreciated! Thanks very much!
> >>
> >> Regards,
> >> Le
> >
> > Clearly, bus level reset can't work for the root bus :)
> 
> Thanks very much!
> I test the vfio with a second-bus ahci controller and it didn't
> complain about the lack of reset mechanism. :) And the return val of
> HOST_CTL is normal now (the same as emulated ahci controller).
> However, it still paused when the BIOS tried to write to the HOST_CTL.
> Do you have any idea?
> And we should just test vfio and legacy pci-assignment with second bus
> devices, not considering the root-bus devices?

AHCI seems like a poor choice of device for this work, they typically
don't support any kind of reset and they can be troublesome even for the
L1 assignment.  You really want something with FLR support so that both
the host and L1 guest can potentially reset the device.  That said, you
may still run into bugs with a L1 guest directed FLR.  Thanks,

Alex