Re: [Qemu-devel] [RFC PATCH] target-ppc: enable migration within the same CPU family

[Alexey Kardashevskiy]

On 04/09/2014 12:59 AM, Alexander Graf wrote:
> On 04/08/2014 02:19 PM, Michael Mueller wrote:
>> On Tue, 08 Apr 2014 21:47:39 +1000
>> Alexey Kardashevskiy <address@hidden> wrote:
>>
>>> On 04/08/2014 08:32 PM, Michael Mueller wrote:
>>>> On Tue, 08 Apr 2014 20:04:42 +1000
>>>> Alexey Kardashevskiy <address@hidden> wrote:
>>>>
>>>>> On 04/08/2014 07:47 PM, Michael Mueller wrote:
>>>>>> On Tue, 08 Apr 2014 11:23:14 +1000
>>>>>> Alexey Kardashevskiy <address@hidden> wrote:
>>>>>>
>>>>>>> On 04/08/2014 04:53 AM, Andreas Färber wrote:
>>>>>>>> Am 07.04.2014 05:27, schrieb Alexey Kardashevskiy:
>>>>>>>>> On 04/04/2014 11:28 PM, Alexander Graf wrote:
>>>>>>>>>> On 04/04/2014 07:17 AM, Alexey Kardashevskiy wrote:
>>>>>>>>>>> On 03/24/2014 04:28 PM, Alexey Kardashevskiy wrote:
>>>>>>>>>>>> Currently only migration fails if CPU version is different even
>>>>>>>>>>>> a bit.
>>>>>>>>>>>> For example, migration from POWER7 v2.0 to POWER7 v2.1 fails
>>>>>>>>>>>> because of
>>>>>>>>>>>> that. Since there is no difference between CPU versions which
>>>>>>>>>>>> could
>>>>>>>>>>>> affect migration stream, we can safely enable it.
>>>>>>>>>>>>
>>>>>>>>>>>> This adds a helper to find the closest POWERPC family class
>>>>>>>>>>>> (i.e. first
>>>>>>>>>>>> abstract class in hierarchy).
>>>>>>>>>>>>
>>>>>>>>>>>> This replaces VMSTATE_UINTTL_EQUAL statement with a custom
>>>>>>>>>>>> handler which
>>>>>>>>>>>> checks if the source and destination CPUs belong to the same
>>>>>>>>>>>> family and
>>>>>>>>>>>> fails if they are not.
>>>>>>>>>>>>
>>>>>>>>>>>> This adds a PVR reset to the default value as it will be
>>>>>>>>>>>> overwritten
>>>>>>>>>>>> by VMSTATE_UINTTL_ARRAY(env.spr, PowerPCCPU, 1024).
>>>>>>>>>>>>
>>>>>>>>>>>> Since the actual migration format is not changed by this patch,
>>>>>>>>>>>> @version_id of vmstate_ppc_cpu does not have to be changed either.
>>>>>>>>>>>>
>>>>>>>>>>>> Signed-off-by: Bharata B Rao <address@hidden>
>>>>>>>>>>>> Signed-off-by: Alexey Kardashevskiy <address@hidden>
>>>>>>>>>>> Ping?
>>>>>>>>>> Can't we just always allow migration to succeed? It's a problem
>>>>>>>>>> of the tool
>>>>>>>>>> stack above if it allows migration to an incompatible host, no?
>>>>>>>>> This is not how libvirt works. It simply sends the source XML,
>>>>>>>>> reconstructs
>>>>>>>>> a guest on the destination side and then migrates. hoping that the
>>>>>>>>> migration will fail is something (which only QEMU has knowledge
>>>>>>>>> of) is
>>>>>>>>> incompatible. The new guest will start with "-cpu host" (as the
>>>>>>>>> source) but
>>>>>>>>> it will create diffrent CPU class and do different things. If we
>>>>>>>>> do not
>>>>>>>>> check PVR (and cpu_dt_id and chip_id - the latter is coming soon) and
>>>>>>>>> migrate power8->power7, we can easily get a broken guest.
>>>>>>>> The response is very simple: -cpu host is not supported for migration.
>>>>>>>> Same as for x86 hosts.
>>>>>>> Is there any good reason to limit ourselves on POWERPC?
>>>>>>>
>>>>>>>> As you say, the domain config is transferred by libvirt:
>>>>>>>> If you use -cpu POWER7, you can migrate from POWER7 to POWER8 and
>>>>>>>> back;
>>>>>>>> if you use -cpu POWER8, you can only migrate on POWER8.
>>>>>>> -cpu other that "host" is not supported by HV KVM, only "compat" which
>>>>>>> upstream QEMU does not have yet. So you are saying that the
>>>>>>> migration is
>>>>>>> not supported by upstream QEMU for at least SPAPR. Well, ok, it is dead
>>>>>>> anyway so I am fine :)
>>>>>>>
>>>>>> With s390x we have a similar situation. Thus we came up with a
>>>>>> mechanism to limit
>>>>>> the CPU functionality of a possible target system. Our patch
>>>>>> implements CPU models
>>>>>> based on TYPE and GA like 2817-ga1, etc. (GA represents a CPU
>>>>>> facility set and an IBC
>>>>>> value (Instruction Blocking Control, reduces the instruction set to
>>>>>> the requested
>>>>>> level)) When a guest is started, it receives its CPU model by means
>>>>>> of option -cpu.
>>>>>> "host" equates the configuration of the current system. We
>>>>>> implemented "query-cpu-model"
>>>>>> returning the actual model, here maybe { name: "2817-ga1" }. To find
>>>>>> a suitable
>>>>>> migration target in a remote CEC, libvirt has to
>>>>>> "query-cpu-definitions" returning a
>>>>>> list of models supported by the target system "{{name: "2827-ga2"},
>>>>>> {name: "2827-ga1"},
>>>>>> {name: "2817-ga2"},...]. A match means the system is suitable and can
>>>>>> be used
>>>>>> as migration target.
>>>>> Sorry, I do not follow you. You hacked libvirt to run the destination
>>>>> QEMU
>>>>> with a specific CPU model? Or it is in QEMU? Where? What I see now is
>>>>> this:
>>>>>
>>>>> static const VMStateDescription vmstate_s390_cpu = {
>>>>>      .name = "cpu",
>>>>>      .unmigratable = 1,
>>>>> };
>>>>>
>>>>> Does not look like it supports migration :) Thanks!
>>>>>
>>>> The code you're missing is not upstream yet. The s390x guest can be
>>>> migrated in the meantime.
>>>> Yes, libvirt currently gets an extension to be able to identify and
>>>> startup suitable migration
>>>> targets for s390x on behalf of the mentioned qemu cpu model. BTW can
>>>> you point me to the above
>>>> mentioned SPAPR stuff...
>>>
>>> Mmm. What stuff? :) At the moment POWERPC guests migrate if PVR (processor
>>> version register) value is exactly the same. I am trying to relax this
>>> limitation to any version within same CPU family, like power7 v1.0 and
>>> v2.1.
>> With stuff I referred to to term sPAPR not realizing it relates to
>> the Power Architecture Platform Requirements, got it now. :-)
>>
>> I see, ppc currently has this limitation to enforce compatibility
>> VMSTATE_UINTTL_EQUAL(env.spr[SPR_PVR], PowerPCCPU),
> 
> Yes, but the s390 approach is a lot cleaner and I'd rather like to move
> into that direction.

Then we will need to support (again) -cpu power7/8 for HV KVM. At the
moment POWER8 is an alias to POWER8_v1.0 and if we try -cpu POWER8 on
versions other than 1.0, HV KVM will fail on attempt to set PVR as it must
be the same as the host PVR (the reason is HV KVM does not emulate PVR, you
know).

To do this, we either need to add every single CPU version to QEMU (very,
very annoying) or allow -cpu to accept CPU family names (which is not
possible now as they are "abstract") or add a version-less CPU class to
every CPU family and redefine aliases to these new CPUs.

Or I do not understand s390 approach, then please correct me :)

What do I miss and what do I have to do now? Thanks!


-- 
Alexey