Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexander Graf]

> Am 09.04.2014 um 02:41 schrieb Alexey Kardashevskiy <address@hidden>:
> 
>> On 04/09/2014 12:59 AM, Alexander Graf wrote:
>>> On 04/08/2014 02:19 PM, Michael Mueller wrote:
>>> On Tue, 08 Apr 2014 21:47:39 +1000
>>> Alexey Kardashevskiy <address@hidden> wrote:
>>> 
>>>>> On 04/08/2014 08:32 PM, Michael Mueller wrote:
>>>>> On Tue, 08 Apr 2014 20:04:42 +1000
>>>>> Alexey Kardashevskiy <address@hidden> wrote:
>>>>> 
>>>>>>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>>>>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>>>>>> Alexey Kardashevskiy <address@hidden> wrote:
>>>>>>> 
>>>>>>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>>>>>> Currently only migration fails if CPU version is different even
>>>>>>>>>>>>> a bit.
>>>>>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails
>>>>>>>>>>>>> because of
>>>>>>>>>>>>> that. Since there is no difference between CPU versions which
>>>>>>>>>>>>> could
>>>>>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This adds a helper to find the closest POWERPC family class
>>>>>>>>>>>>> (i.e. first
>>>>>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom
>>>>>>>>>>>>> handler which
>>>>>>>>>>>>> checks if the source and destination CPUs belong to the same
>>>>>>>>>>>>> family and
>>>>>>>>>>>>> fails if they are not.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> This adds a PVR reset to the default value as it will be
>>>>>>>>>>>>> overwritten
>>>>>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Signed-off-by: Bharata B Rao <address@hidden>
>>>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <address@hidden>
>>>>>>>>>>>> Ping?
>>>>>>>>>>> Can't we just always allow migration to succeed? It's a problem
>>>>>>>>>>> of the tool
>>>>>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>>>>> This is not how libvirt works. It simply sends the source XML,
>>>>>>>>>> reconstructs
>>>>>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>>>>>> migration will fail is something (which only QEMU has knowledge
>>>>>>>>>> of) is
>>>>>>>>>> incompatible. The new guest will start with "-cpu host" (as the
>>>>>>>>>> source) but
>>>>>>>>>> it will create diffrent CPU class and do different things. If we
>>>>>>>>>> do not
>>>>>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>>>>>> Same as for x86 hosts.
>>>>>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>>>>> 
>>>>>>>>> As you say, the domain config is transferred by libvirt:
>>>>>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and
>>>>>>>>> back;
>>>>>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>>>>>> upstream QEMU does not have yet. So you are saying that the
>>>>>>>> migration is
>>>>>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>>>>>> anyway so I am fine :)
>>>>>>> With s390x we have a similar situation. Thus we came up with a
>>>>>>> mechanism to limit
>>>>>>> the CPU functionality of a possible target system. Our patch
>>>>>>> implements CPU models
>>>>>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU
>>>>>>> facility set and an IBC
>>>>>>> value (Instruction Blocking Control, reduces the instruction set to
>>>>>>> the requested
>>>>>>> level)) When a guest is started, it receives its CPU model by means
>>>>>>> of option -cpu.
>>>>>>> "host" equates the configuration of the current system. We
>>>>>>> implemented "query-cpu-model"
>>>>>>> returning the actual model, here maybe { name: "2817-ga1" }. To find
>>>>>>> a suitable
>>>>>>> migration target in a remote CEC, libvirt has to
>>>>>>> "query-cpu-definitions" returning a
>>>>>>> list of models supported by the target system "{{name: "2827-ga2"},
>>>>>>> {name: "2827-ga1"},
>>>>>>> {name: "2817-ga2"},...]. A match means the system is suitable and can
>>>>>>> be used
>>>>>>> as migration target.
>>>>>> Sorry, I do not follow you. You hacked libvirt to run the destination
>>>>>> QEMU
>>>>>> with a specific CPU model? Or it is in QEMU? Where? What I see now is
>>>>>> this:
>>>>>> 
>>>>>> static const VMStateDescription vmstate_s390_cpu = {
>>>>>>     .name = "cpu",
>>>>>>     .unmigratable = 1,
>>>>>> };
>>>>>> 
>>>>>> Does not look like it supports migration :) Thanks!
>>>>> The code you're missing is not upstream yet. The s390x guest can be
>>>>> migrated in the meantime.
>>>>> Yes, libvirt currently gets an extension to be able to identify and
>>>>> startup suitable migration
>>>>> targets for s390x on behalf of the mentioned qemu cpu model. BTW can
>>>>> you point me to the above
>>>>> mentioned SPAPR stuff...
>>>> 
>>>> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
>>>> version register) value is exactly the same. I am trying to relax this
>>>> limitation to any version within same CPU family, like power7 v1.0 and
>>>> v2.1.
>>> With stuff I referred to to term sPAPR not realizing it relates to
>>> the Power Architecture Platform Requirements, got it now. :-)
>>> 
>>> I see, ppc currently has this limitation to enforce compatibility
>>> VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
>> 
>> Yes, but the s390 approach is a lot cleaner and I'd rather like to move
>> into that direction.
> 
> Then we will need to support (again) -cpu power7/8 for HV KVM. At the
> moment POWER8 is an alias to POWER8_v1.0 and if we try -cpu POWER8 on
> versions other than 1.0, HV KVM will fail on attempt to set PVR as it must
> be the same as the host PVR (the reason is HV KVM does not emulate PVR, you
> know).
> 
> To do this, we either need to add every single CPU version to QEMU (very,
> very annoying) or allow -cpu to accept CPU family names (which is not
> possible now as they are "abstract") or add a version-less CPU class to
> every CPU family and redefine aliases to these new CPUs.

Yes, I think -cpu POWER8 should be an alias to 1.0 for TCG, but work within the 
constraints of the p8 masks the same way as -cpu host for KVM.


Alex

> 
> Or I do not understand s390 approach, then please correct me :)
> 
> What do I miss and what do I have to do now? Thanks!
> 
> 
> -- 
> Alexey