qemu-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

virtio-iommu hotplug issue


From: Akihiko Odaki
Subject: virtio-iommu hotplug issue
Date: Thu, 13 Apr 2023 13:49:43 +0900
User-agent: Mozilla/5.0 (X11; Linux aarch64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0

Hi,

Recently I encountered a problem with the combination of Linux's virtio-iommu driver and QEMU when a SR-IOV virtual function gets disabled. I'd like to ask you what kind of solution is appropriate here and implement the solution if possible.

A PCIe device implementing the SR-IOV specification exports a virtual function, and the guest can enable or disable it at runtime by writing to a configuration register. This effectively looks like a PCI device is hotplugged for the guest. In such a case, the kernel assumes the endpoint is detached from the virtio-iommu domain, but QEMU actually does not detach it.

This inconsistent view of the removed device sometimes prevents the VM from correctly performing the following procedure, for example:
1. Enable a VF.
2. Disable the VF.
3. Open a vfio container.
4. Open the group which the PF belongs to.
5. Add the group to the vfio container.
6. Map some memory region.
7. Close the group.
8. Close the vfio container.
9. Repeat 3-8

When the VF gets disabled, the kernel assumes the endpoint is detached from the IOMMU domain, but QEMU actually doesn't detach it. Later, the domain will be reused in step 3-8.

In step 7, the PF will be detached, and the kernel thinks there is no endpoint attached and the mapping the domain holds is cleared, but the VF endpoint is still attached and the mapping is kept intact.

In step 9, the same domain will be reused again, and the kernel requests to create a new mapping, but it will conflict with the existing mapping and result in -EINVAL.

This problem can be fixed by either of:
- requesting the detachment of the endpoint from the guest when the PCI device is unplugged (the VF is disabled) - detecting that the PCI device is gone and automatically detach it on QEMU-side.

It is not completely clear for me which solution is more appropriate as the virtio-iommu specification is written in a way independent of the endpoint mechanism and does not say what should be done when a PCI device is unplugged.

Regards,
Akihiko Odaki



reply via email to

[Prev in Thread] Current Thread [Next in Thread]