Re: [Qemu-devel] [PATCH 00/13] IOMMU: Enable interrupt remapping for Intel IOMMU

[Peter Xu]

On Fri, Feb 19, 2016 at 09:34:40AM +0100, Jan Kiszka wrote:
> On 2016-02-19 08:43, Peter Xu wrote:
> > Hi, Jan,
> > 
> > On Fri, Feb 19, 2016 at 07:46:26AM +0100, Jan Kiszka wrote:
> >> Hi Peter,
> >>
> >> On 2016-02-19 04:30, Peter Xu wrote:
> >>> This patchset provide very basic functionalities for interrupt
> >>> remapping (IR) support of the emulated Intel IOMMU device.
> >>
> >> Interesting. Some questions:
> >>
> >> - Were you aware of
> >> http://git.kiszka.org/?p=qemu.git;a=shortlog;h=refs/heads/queues/vtd-intremap,
> >> and can you comment on how this relates to your approach? Mine included
> >> HPET support, e.g.
> > 
> > No. I do not know we have existing works related. I'd say that I am
> > one of those who are not fan of duplicating works. If I know that
> > there is existing ones, I would have asked you first.
> > 
> > Actually there are several people within my working team knows that
> > I would be working on this, and I believe none of us do know your
> > work too... Or there must be someone telling me so...
> 
> I guess we would have to match sets of people know to find out who
> forgot to mention the outreachy project ;) - anyway, this can always happen.
> 
> > 
> > Whatever, I am sorry that finally we got some duplicated work. :-(
> > 
> > I have not go deeper into your codes yet, I'd say that most of the
> > logic is alike, since lots of works are to follow the spec. And the
> > implementation is slightly different (actually I just got this
> > memory region idea from Paolo yesterday... instead of a worse one to
> > declare a specific func ops for MSIs).
> 
> I didn't look into your approach in all details yet, and Rita also just
> started, she told me. So one question on yours - which looks appealing
> from the invasiveness POV - is how you determine the MSI source with
> only a single target region? I do find your changes on the IOAPIC, but
> none on the PCI infrastructure, which is confusing given that you say
> that works, no?

Do we need to know the source of the MSI interrupt to
translate/deliver it? Maybe I got something missing, but could you
explain why we need it?

What I understand is that, for interrupt remapping, IRT (Interrupt
Remapping Table) is shared within the whole device tree managed by
specific IOMMU (or say, IRE is shared between different source IDs),
which is kinda different from DMAR.

The patch should basically work at least on my machine, and please
feel free to try it if you want. I just created one public tree on
github, link will be pasted below. Also please refer to the cover
letter for the command line to boot the guest. I have not done any
thorough tests, since I just want to first post it out in case there
is anything not right, and both luckily & unluckily... I got to know
that there are people working on it before going deeper. :(

> 
> My design introduces a per-source MSI (DMA) target region so that the
> IOMMU can do proper remapping by deriving the source device ID from the
> targeted region. Then you only need to give the special sources like
> IOAPIC and HPET their only regions, and you are done.

My thought explained as above. Please let me know if I was wrong.

> 
> > 
> > What I was going to do is to at least support:
> > 
> > - vhost
> > - pass through devices
> 
> vhost, i.e. virtio, has still problems as current virtio guest drivers
> assume that their devices are not under control of any IOMMU unit. I
> didn't trace the latest news here, but we were once discussing to
> initially exclude virtio devices in DMAR scope descriptions in ACPI
> and/or introduce some feature negotiation between guest and host to
> identify and reject legacy guest drivers without IOMMU-awareness.
> 
> > 
> > So what I was planning is that, this series will be the start. And
> > the above two is the first-step goal (and I may need kvm-ioapic as
> > well though).
> 
> KVM support is actually a key thing, that's why we started the project
> on integrating the patches with the split irqchip work. There was a
> consensus with Paolo long ago that full in-kernel irqchip will no longer
> gain any additional support that is needed for IR.

Do you have any link to the discussion? I am just curious about it,
thanks in advance.

> 
> > 
> > With the above, we should be able to run DPDK applications in guests
> > with either pass-through or vhost-user devices (these should be the
> > two scenario which is most possibly to be used AFAIU). This is what
> > I was trying to do. HPET is not with high priority in my todo list.
> > 
> >>
> >> - Rita Sinha is currently working on integrating my old patches with the
> >> split-irqchip to get KVM working (as an Outreachy project). It's
> >> probably a bit unfortunate to consider a different horse that late in to
> >> project. What do you think, how could we benefit from each other?
> > 
> > I'd say that I am still new to QEMU community. So this is the first
> > time I encountered this kind of problem... Do you have any
> > suggestion?
> > 
> >>
> >> - Radim was telling me to look on this as well. How do your efforts
> >> correlate?
> > 
> > Same as above. Do you have any suggestion on how we'd move on?
> 
> Specifically you and Rita should exchange your knowledge on both designs
> (here on the list, of course) and try to find an optimal path. We are
> unfortunately a bit time-constrained /wrt Outreachy as that project ends
> in a couple of weeks. By then, we should have first results for the KVM
> integration - not necessarily the perfect ones, but something working
> and following the "right" direction.

Regarding to the current KVM work for this, do we have discussion
thread or link so that I can know more about?

I see that the project does there:

http://qemu-project.org/Outreachy_2015_MayAugust#VT-d_interrupt_emulation_with_KVM_support

Sorry that I failed to notice it before. No matter which to choose, I would
like to help in any way.

> 
> > 
> > One question about your tree: have you posted patches before? and
> > what's the relationship between your tree and current QEMU master?
> 
> I never posted the patches as they never left the state of "hacks" as
> you can quickly spot. However, I think the design is not a hack, but
> maybe there is indeed something even smarter.
> 
> Rita has rebased the patches recently, maybe you can share your state,
> Rita? I did this before, because I'm using them since day #1 to do
> regular testing of our Jailhouse hypervisor inside QEMU/KVM. For that
> purpose they work very reliably for IOAPIC, HPET and emulated PCI devices.
> 
[...]
> 
> Would be helpful for more convenient import and experiments. E.g. I
> could quickly through my test setup at them and tell you if they work
> for it.

I have put codes onto github for better reference:

https://github.com/xzpeter/qemu/tree/vt-d-intr

Thanks.
Peter

> 
> Jan
> 
>