Re: [RFC v3] VFIO Migration

[Daniel P . Berrangé]

On Mon, Nov 16, 2020 at 07:34:25AM -0500, Michael S. Tsirkin wrote:
> On Mon, Nov 16, 2020 at 12:05:18PM +0000, Daniel P. Berrangé wrote:
> > On Mon, Nov 16, 2020 at 07:03:03AM -0500, Michael S. Tsirkin wrote:
> > > On Mon, Nov 16, 2020 at 11:41:25AM +0000, Daniel P. Berrangé wrote:
> > > > > It is possible to simplify the problem, but we'll lose freedom. For
> > > > > example, hard coding knowledge of the device implementation into the
> > > > > management tool eliminates the need for a general migration checking
> > > > > algorithm. Or we might be able to simplify it by explicitly not
> > > > > supporting cross-device implementation migration (although that would
> > > > > place stricter rules on what a new version of an existing device can
> > > > > change in order to preserve migration compatibility).
> > > > 
> > > > Is migrating between 2 different vendors' impls of the same core
> > > > device spec really a thing that's needed ? 
> > > 
> > > If there's intent to have this supercede vhost-user then certainly.
> > > Same I'm guessing for NVMe.
> > > 
> > > 
> > > > > I have doubts that these trade-offs can be made without losing support
> > > > > for use cases that are necessary.
> > > > 
> > > > >From my POV, the key goal is that it should be possible to migrate
> > > > between two hosts without needing to check every single possible
> > > > config parameter that the device supports. It should only be neccessary
> > > > to check the parameters that are actually changed from their default
> > > > values. Then there just needs to be some simple string parameter that
> > > > encodes a particular set of devices, akin to the versioned machine
> > > > type.
> > > > 
> > > > Applications that want to migration between cross-vendor device impls
> > > > could opt-in to checking every single little parameter, but most can
> > > > just stick with a much simplified view where they only have to check
> > > > the parameters that they've actually overriden/exposed.
> > > 
> > > It's a problem even for a single vendor. And we have lots of experience
> > > telling us it's a messy, difficult one. Just punting and saying
> > > vendors will do the right thing will not lead to quality
> > > implementations.
> > 
> > I'm not suggesting we punt on the problem. I'm saying that checking for
> > migration compatibility should not need to be made more complex than what
> > we already do for QEMU. The core problem being tackled is essentially the
> > same in both cases.
> 
> There's a difference: in case of QEMU versions are release based.  At
> release time a new version is generated.  So QEMU upstream ships version
> X and Red Hat ships Y at a different time and they are not compatible.

That's a difference that Red Hat maintainers chose to introduce. RHEL
could have stuck with upstream QEMU machine types if it wished to, but
it chose to ship different machine types, because it made life easier
to backport features that impacted machine types, and also to some extent
to let us fix migration compat screw ups. We could have stuck to upstream
machine types though and remained compatible.  Many other distros do just
that.

> This won't work for devices: same device needs to work with
> both upstream and Red Hat and migrate upstream-upstream and Red Hat-Red Hat
> (though not upstream-Red Hat).

That's fine, we can cope with that. It simply means whomever owns
responsibility for maintaining the code has to be more careful about
changes they make in their downstream.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|