Re: [Qemu-devel] kvm PCI assignment & VFIO ramblings

[Alexander Graf]

On 26.08.2011, at 04:33, Roedel, Joerg wrote:

> On Fri, Aug 26, 2011 at 12:20:00AM -0400, David Gibson wrote:
>> On Wed, Aug 24, 2011 at 01:03:32PM +0200, Roedel, Joerg wrote:
>>> On Wed, Aug 24, 2011 at 05:33:00AM -0400, David Gibson wrote:
>>>> On Wed, Aug 24, 2011 at 11:14:26AM +0200, Roedel, Joerg wrote:
>>> 
>>>>> I don't see a reason to make this meta-grouping static. It would harm
>>>>> flexibility on x86. I think it makes things easier on power but there
>>>>> are options on that platform to get the dynamic solution too.
>>>> 
>>>> I think several people are misreading what Ben means by "static".  I
>>>> would prefer to say 'persistent', in that the meta-groups lifetime is
>>>> not tied to an fd, but they can be freely created, altered and removed
>>>> during runtime.
>>> 
>>> Even if it can be altered at runtime, from a usability perspective it is
>>> certainly the best to handle these groups directly in qemu. Or are there
>>> strong reasons to do it somewhere else?
>> 
>> Funny, Ben and I think usability demands it be the other way around.
> 
> The reason is that you mean the usability for the programmer and I mean
> it for the actual user of qemu :)

No, we mean the actual user of qemu. The reason being that making a device 
available for any user space application is an administrative task.

Forget the KVM case for a moment and think of a user space device driver. I as 
a user am not root. But I as a user when having access to /dev/vfioX want to be 
able to access the device and manage it - and only it. The admin of that box 
needs to set it up properly for me to be able to access it.

So having two steps is really the correct way to go:

  * create VFIO group
  * use VFIO group

because the two are done by completely different users. It's similar to how 
tun/tap works in Linux too. Of course nothing keeps you from also creating a 
group on the fly, but it shouldn't be the only interface available. The 
persistent setup is definitely more useful.

> 
>> If the meta-groups are transient - that is lifetime tied to an fd -
>> then any program that wants to use meta-groups *must* know the
>> interfaces for creating one, whatever they are.
>> 
>> But if they're persistent, the admin can use other tools to create the
>> meta-group then just hand it to a program to use, since the interfaces
>> for _using_ a meta-group are identical to those for an atomic group.
>> 
>> This doesn't preclude a program from being meta-group aware, and
>> creating its own if it wants to, of course.  My guess is that qemu
>> would not want to build its own meta-groups, but libvirt probably
>> would.
> 
> Doing it in libvirt makes it really hard for a plain user of qemu to
> assign more than one device to a guest. What I want it that a user just
> types
> 
>       qemu -device vfio,host=00:01.0 -device vfio,host=00:02.0 ...
> 
> and it just works. Qemu creates the meta-groups and they are
> automatically destroyed when qemu exits. That the programs are not aware
> of meta-groups is not a big problem because all software using vfio
> needs still to be written :)
> 
> Btw, with this concept the programmer can still decide to not use
> meta-groups and just multiplex the mappings to all open device-fds it
> uses.

What I want to see is:

  # vfio-create 00:01.0
    /dev/vfio0
  # vftio-create -a /dev/vfio0 00:02.0
    /dev/vfio0

  $ qemu -vfio dev=/dev/vfio0,id=vfio0 -device vfio,vfio=vfio0.0 -device 
vfio,vfio=vfio0.1


Alex