[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] QEMU and vIOMMU support for emulated VF passthrough to
From: |
Peter Xu |
Subject: |
Re: [Qemu-devel] QEMU and vIOMMU support for emulated VF passthrough to nested (L2) VM |
Date: |
Wed, 27 Mar 2019 14:41:43 +0800 |
User-agent: |
Mutt/1.10.1 (2018-07-13) |
On Tue, Mar 26, 2019 at 01:23:12PM +0000, Elijah Shakkour wrote:
> Adding QEMU-devel
Hi, Elijah,
>
> -----Original Message-----
> From: Michael S. Tsirkin <address@hidden>
> Sent: Tuesday, March 26, 2019 2:53 PM
> To: Elijah Shakkour <address@hidden>
> Cc: Knut Omang <address@hidden>; Alex Williamson <address@hidden>; Marcel
> Apfelbaum <address@hidden>; Stefan Hajnoczi <address@hidden>; address@hidden
> Subject: Re: QEMU and vIOMMU support for emulated VF passthrough to nested
> (L2) VM
>
> I think you forgot to copy the qemu mailing list.
>
> On Tue, Mar 26, 2019 at 10:08:17AM +0000, Elijah Shakkour wrote:
> > My questions are:
> >
> > - Suppose that there is an emulated NIC that supports SRIOV (I implemented
> > such a NIC), now does QEMU support a scenario of an emulated NIC that
> > supports SRIOV in Hyper-V L1 guest, that invokes VF and pass it to nested
> > linux L2 guest?
I am not an expert of SR-IOV but I can't see a limitation to not allow
that to happen.
> > - I'm using vIOMMU in L1, so what is needed to be done in QEMU or maybe in
> > emulated NIC PF/VF to allow DMA remapping and INT remapping work as
> > expected?
Your below command line should work, and even it seems to be an
overkill.
If your device is completely emulated, IIUC you only simply need this
on the latest QEMU:
-M q35 -device intel-iommu
Split-irqchip and IR is on by default now, so you'll naturally gain
x2apic if it's supported. You can use x-aw-bits but only if you
really need address space beyond 39 bits (which I suspect). The rest
parameters are optional too.
> > - Does the command line below -that I use to run QEMU- seem ok for the
> > scenario I described to work?
Before I look into details of the cmdline - I'd say MMIO in L2 should
have nothing to do with IOMMU... Are you sure the MMIO traps are
setup correctly? Can the VF do IO properly even without L2?
Also I don't know whether there can be some tricks when you boot L2
with vfio-pci when the device to assign is a VF.
> >
> > -----Original Message-----
> > From: Michael S. Tsirkin <address@hidden>
> > Sent: Monday, March 25, 2019 4:14 AM
> > To: Elijah Shakkour <address@hidden>
> > Cc: Knut Omang <address@hidden>; Alex Williamson
> > <address@hidden>; Marcel Apfelbaum
> > <address@hidden>; Stefan Hajnoczi <address@hidden>
> > Subject: Re: QEMU and vIOMMU support for emulated VF passthrough to
> > nested (L2) VM
> >
> > Pls post all questions on list.
> > I have a policy against answering off-list mail.
> > Cc Pter Xu might be a good idea, too.
> >
> > On Sun, Mar 24, 2019 at 09:56:26PM +0000, Elijah Shakkour wrote:
> > > Hey,
> > >
> > > I'm emulating Mellanox ConnectX-4 in QEMU and right now, I'm adding SRIOV
> > > capability.
> > > I'm using Knut Omang SRIOV patches rebased to QEMU v2.12.
> > > My server (L0) is Linux. L1 guest is Windows2016 Hyper-V and L2 guest is
> > > Linux RH7.2.
> > > I can see my device in L1 VM and I see the invocation of the VF via SRIOV
> > > capability.
> > > Inside L2 guest I see the virtual function in "lspci' command.
> > > But when driver of L2 guest issues MMIO read/write, my MMIO ops don't get
> > > called.
> > > I implemented my VF basically like Omang SRIOV example patch.
> > >
> > > Could you please shed some light on what you think I might be missing?
> > >
> > > Here is the command line I run:
> > >
> > > ./x86_64-softmmu/qemu-system-x86_64 \ -machine
> > > q35,accel=kvm,usb=off,dump-guest-core=off,kernel-irqchip=split \ -m
> > > 32G \ -smp 2 \ -enable-kvm \ -cpu
> > > host,vmx=on,ss=on,cx16=on,x2apic=on,hypervisor=on,lahf_lm=on,hv_time
> > > ,h v_relaxed,hv_vapic,hv_spinlocks=0x1fff,kvm=on \ -vnc
> > > 127.0.0.1:0,to=99,id=default \ -drive
> > > file=$IMAGE,format=qcow2,if=none,id=drive-sata0-0-0 \ -chardev
> > > pty,id=charserial0 \ -device
> > > intel-iommu,intremap=on,caching-mode=on,device-iotlb=on,eim=on,x-aw-
> > > bi
> > > ts=48 \ -device
> > > ide-hd,bus=ide.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=0 \
> > > -device
> > > pcie-root-port,pref64-reserve=500M,slot=0,id=pcie_port.1,bus=pcie.0,
> > > mu
> > > ltifunction=on \ -netdev
> > > tap,id=tap5,ifname=tap5,script=no,downscript=no \ -device
> > > connectx4,netdev=tap5,bus=pcie_port.1,multifunction=on
Regards,
--
Peter Xu