Re: [Qemu-devel] [Query] Live Migration between machines with different processor ids

[Andrew Jones]

Manish,

Your mail reader doesn't appear to be providing any quoting, so I'm not
sure I found all your replies. I've tried to fix up the quoting in order
to reply here, but you should look into fixing your reader.

On Fri, Aug 31, 2018 at 09:52:12AM +0000, Jaggi, Manish wrote:
> On bleh-bleh, Andrew Jones wrote:
> > On bleh-bleh, Jaggi, Manish wrote:
> > > - So in cpu_post_load (Machine B) qemu can lookup whitelist and replace 
> > > the MIDR with the one at Machine B.
> > > Sounds good?
> > > 
> > It shouldn't be necessary. With '-cpu host' QEMU should probably just read
> > all the ID registers from the host first, updating the guest's copy to
> > match the destination host's registers (we're using '-cpu host', the
> > registers should match the host - including MIDR.) If a user chooses to
> > migrate a guest that is using '-cpu host', then they need to know what
> > they are doing. If a whitelist of close-enough processors is possible to
> > create, then that whitelist should be managed and used at a higher layer
> > in the virt stack, not down in QEMU.
> 
> How would qemu know to that it has to patch? Could not understand your point 
> here.
> IIUC qemu needs some parameter for this

QEMU would unconditionally update the guest's view of the invariant
registers after migration, when '-cpu host' is used. There's no need
for a parameter, as the update is unconditional.

> 
> > For example, openstack can determine
> > destination candidates using whatever policy it wants, including a close-
> > enough processor whitelist.
> > 
> > So, I propose blindly updating all invariant registers when migrating
> > a '-cpu host' guest and leaving it to the user to do these migrations
> > at their own risk
> > 
> yes good point updating all invariant is better, for the case where user is 
> aware that host and destination have same cpu arch.
> I can prepare a RFC patch but cpu_post_load will need some flag to replace 
> these values.,

I'm not sure what you mean by a flag, but I'll review the RFC when you
post it.

> > 
> > (when migrating to a truly identical host, the blind
> > update will not change anything. So it would be no worse than what we
> > do today.) One side note is that we're starting to give QEMU control
> > over what optional processor features are available to the guest, e.g.
> > SVE. So before blindly updating all ID registers we'd want to inform
> > KVM of the guest configuration in order for KVM to return appropriate
> > ID register values.
> > 
> Not sure how to handle this.

I think the sequence should look something like this:

  1) Guest running on Host A with processor a
  2) Stop guest and save its state for migration
  3) Migrate guest to Host B with processor b (b is "close enough" to a)
  4) Restore guest state after migration
     If guest is running with '-cpu host'
       4.a) Inform KVM of any configuration that impacts invariant registers
       4.b) Update the guest's view of all invariant registers to match the
            host
     EndIf
  5) Run guest

4.a and 4.b require new code both in QEMU and KVM. 4.a may require a new
KVM API, unless the existing API can be leveraged.

The definition of "close enough" is left to the users and/or higher layers
of the Virt stack.

Thanks,
drew