Re: [Qemu-devel] [PATCH v2 02/19] spapr: introduce a skeleton for the XIVE interrupt controller

[Cédric Le Goater]

On 04/16/2018 06:29 AM, David Gibson wrote:
> On Thu, Apr 12, 2018 at 10:36:10AM +0200, Cédric Le Goater wrote:
>> On 04/12/2018 07:16 AM, David Gibson wrote:
>>> On Mon, Feb 12, 2018 at 09:55:17AM +1100, Benjamin Herrenschmidt wrote:
>>>> On Sun, 2018-02-11 at 19:08 +1100, David Gibson wrote:
>>>>> On Thu, Jan 18, 2018 at 08:27:52AM +1100, Benjamin Herrenschmidt wrote:
>>>>>> On Wed, 2018-01-17 at 15:39 +0100, Cédric Le Goater wrote:
>>>>>>> Migration is a problem. We will need both backend QEMU objects to be 
>>>>>>> available anyhow if we want to migrate. So we are back to the current 
>>>>>>> solution creating both QEMU objects but we can try to defer some of the 
>>>>>>> KVM inits and create the KVM device on demand at CAS time.
>>>>>>
>>>>>> Do we have a way to migrate a piece of info from the machine *first*
>>>>>> that indicate what type of XICS/XIVE to instanciate ?
>>>>>
>>>>> Nope.  qemu migration doesn't work like that.  Yes, it should, and
>>>>> everyone knows it, but changing it is a really long term project.
>>>>
>>>> Well, we have a problem then. It looks like Qemu broken migration is
>>>> fundamentally incompatible with PAPR and CAS design...
>>>
>>> Hrm, the fit is very clunky certainly, but i think we can make it work.
>>>
>>>> I know we don't migrate the configuration, that's not exactly what I
>>>> had in mind tho... Can we have some piece of *data* from the machine be
>>>> migrated first, and use it on the target to reconfigure the interrupt
>>>> controller before the stream arrives ?
>>>
>>> Sorta.. maybe.. but it would probably get really ugly if we don't
>>> preserve the usual way object lifetimes work.
>>>
>>>> Otherwise, we have indeed no much choice but the horrible wart of
>>>> creating both interrupt controllers with only one "active".
>>>
>>> I really think this is the way to go, warts and all.
>>>
>>
>> Yes ... KVM makes it a little uglier. 
>>
>> A KVM_DEVICE_DESTROY device is needed to cleanup the VM and a 
>> DISABLE_CAP to disconnect the vpcu from the current KVM XIVE/XICS 
>> device. I have used an extra arg on ENABLE_CAP for the moment.    
>>
>> At the QEMU level, we need to connect/reconnect at reset time to
>> handle possible changes in CAS, and at post_load.
> 
> Right.

v3 uses the same 'reset' function to setup the interrupts at machine 
reset time and at post_load. Keep that in mind. May be we should have 
distinct routines.
 
> 
>> Destroying the MemoryRegion is a bit problematic, I have not
>> found a common layout compatible with both the emulated mode 
>> (std IO regions) and the KVM mode (ram device regions)
> 
> That sounds awkward, I guess we'll discuss the details of this later.

I have fixed that in v3.

> Btw, a secondary advantage of starting off with XIVE only under a
> different machine type is that we can declare that one not to be
> migration stable until we're ready.  So we can merge something that's
> ok to experiment with, but reserve the right to incompatibly change
> the migration format until we're confident we're ready and can merge
> it into the "stable" machine type.

Reseting KVM devices is not the most complex feature to support but 
it seems to have an impact on migration. So we might need the 
non-migratable machine type to fix that. Let's see how v3 is welcomed
or not.
  
 
Thanks,

C.