[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 2/2] Do not access /dev/mem in MSI-X PCI passthrough on Xen
From: |
Marek Marczykowski-Górecki |
Subject: |
Re: [PATCH 2/2] Do not access /dev/mem in MSI-X PCI passthrough on Xen |
Date: |
Wed, 16 Nov 2022 22:40:02 +0100 |
On Wed, Nov 16, 2022 at 02:15:22PM -0500, Jason Andryuk wrote:
> On Mon, Nov 14, 2022 at 2:21 PM Marek Marczykowski-Górecki
> <marmarek@invisiblethingslab.com> wrote:
> >
> > The /dev/mem is used for two purposes:
> > - reading PCI_MSIX_ENTRY_CTRL_MASKBIT
> > - reading Pending Bit Array (PBA)
> >
> > The first one was originally done because when Xen did not send all
> > vector ctrl writes to the device model, so QEMU might have outdated old
> > register value. This has been changed in Xen, so QEMU can now use its
> > cached value of the register instead.
> >
> > The Pending Bit Array (PBA) handling is for the case where it lives on
> > the same page as the MSI-X table itself. Xen has been extended to handle
> > this case too (as well as other registers that may live on those pages),
> > so QEMU handling is not necessary anymore.
> >
> > Removing /dev/mem access is useful to work within stubdomain, and
> > necessary when dom0 kernel runs in lockdown mode.
> >
> > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>
> I put the Xen, QEMU, and xen-pciback patches into OpenXT and gave a
> little test. When pci_permissive=0, iwlwifi fails to load its
> firmware. With pci_permissive=1, it looks like MSI-X is enabled. (I
> previously included your libxl allow_interrupt_control patch - that
> seemed to get regular MSIs working prior to the MSI-X patches.) I
> also removed the OpenXT equivalent of 0005-Disable-MSI-X-caps.patch.
> I am testing with Linux 5.4.y, so that could be another factor.
Can you confirm the allow_interrupt_control is set by libxl? Also,
vanilla 5.4 doesn't have the allow_interrupt_control patch at all, and you
may have an earlier version that had "allow_msi_enable" as the sysfs
file name.
> One strange thing is the lspci output. Dom0 shows MSI-X enabled.
> Meanwhile NDVM (sys-net) does not show the MSI-X capability. If you
> `hexdump -C /sys/bus/pci/devices/$dev/config` you can see MSI-X
> enabled, but you also see that the MSI capability has 00 as the next
> pointer, so lspci stops parsing.
This 00 value is written by Linux[1] (sic!) and then qemu incorrectly
allowing the write and happily emulating that zero. The other qemu patch
in this series ought to fix that (as in: properly refuse the write), do
you have it included?
[1]
https://github.com/torvalds/linux/blob/master/drivers/net/wireless/intel/iwlwifi/pcie/drv.c#L1721
> MSI cap stubdom:
> 00000040 10 00 92 00 c0 0e 00 00 10 0c 10 00 00 00 00 00 |................|
> 0x41 -> next 0x00
> MSI cap dom0:
> 00000040 10 80 92 00 c0 0e 00 10 10 0c 10 00 00 00 00 00 |................|
> 0x41 -> next 0x80
>
> MSI-X:
> 00000080 11 00 0f 80 00 20 00 00 00 30 00 00 00 00 00 00
>
> AFAIU, the value 0x80 at offset 0x83 is MSI-X Enabled.
>
> I had a boot where assignment failed with the hypervisor printing:
> d12: assign (0000:00:14.3) failed (-16)
> Rebooting the laptop seemed to clear that.
Zombie of previous domain? Not set as "assignable" first?
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
signature.asc
Description: PGP signature
Re: [PATCH 1/2] hw/xen/xen_pt: Call default handler only if no custom one is set, Anthony PERARD, 2022/11/22