Re: [Qemu-devel] [RFC PATCH] vfio: VFIO Driver core framework

[Alex Williamson]

On Thu, 2011-12-01 at 14:58 -0600, Stuart Yoder wrote:
> >> The attributes are not intrinsic features of the domain.  User space will
> >> need to set them.  But in thinking about it a bit more I think the 
> >> attributes
> >> are more properties of the domain rather than a per map() operation
> >> characteristic.  I think a separate API might be appropriate.  Define a
> >> new set_domain_attrs() op in the iommu_ops.    In user space, perhaps
> >>  a new vfio group API-- VFIO_GROUP_SET_ATTRS,
> >> VFIO_GROUP_GET_ATTRS.
> >
> > In that case, you should definitely be following what Alexey is thinking
> > about with an iommu_setup IOMMU API callback.  I think it's shaping up
> > to do:
> >
> > x86:
> >  - Report any IOVA range restrictions imposed by hw implementation
> > POWER:
> >  - Request IOVA window size, report size and base
> > powerpc:
> >  - Set domain attributes, probably report range as well.
> 
> One other mechanism we need as well is the ability to
> enable/disable a domain.
> 
> For example-- suppose a device is assigned to a VM, the
> device is in use when the VM is abruptly terminated.  The
> VM terminate would shut off DMA at the IOMMU, but now
> the device is in an indeterminate state.   Some devices
> have no simple reset bit and getting the device back into
> a sane state could be complicated-- something the hypervisor
> doesn't want to do.
> 
> So now KVM restarts the VM, vfio init happens for the device
> and  the IOMMU for that device is re-configured,
> etc, but we really can't re-enable DMA until the guest OS tells us
> (via an hcall) that it is ready.   The guest needs to get the
> assigned device in a sane state before DMA is enabled.

Giant red flag.  We need to paravirtualize the guest?  Not on x86.  Some
devices are better for assignment than others.  PCI devices are moving
towards supporting standard reset mechanisms.

> Does this warrant a new domain enable/disable API, or should
> we make this part of the setup API we are discussing
> here?

What's wrong with simply not adding any DMA mapping entries until you
think your guest is ready?  Isn't that effectively the same thing?
Unmap ~= disable.  If the IOMMU API had a mechanism to toggle the iommu
domain on and off, I wouldn't be opposed to adding an ioctl to do it,
but it really seems like just a shortcut vs map/unmap.  Thanks,

Alex