Re: [Qemu-devel] pseries-2.6 migration from QEMU-2.6 to QEMU-2.7 broken

[Cédric Le Goater]

On 09/22/2016 08:00 AM, Bharata B Rao wrote:
> On Thu, Sep 22, 2016 at 03:30:08PM +1000, David Gibson wrote:
>> On Thu, Sep 22, 2016 at 10:51:05AM +0530, Bharata B Rao wrote:
>>> Hi,
>>>
>>> Nikunj and I realized that migrating pseries-2.6 guest from QEMU-2.6
>>> to newer QEMU-2.7 is broken like this:
>>>
>>> qemu-system-ppc64: error while loading state for instance 0x0 of device 
>>> 'cpu'
>>> qemu-system-ppc64: load of migration failed: Invalid argument
>>>
>>> Bisecting tells us that 4e0806110c8b896ceff3490f15a616e8b3165efe
>>> (ppc: Add PPC_64H instruction flag to POWER7 and POWER8) is the
>>> first bad commit.  Along with this there are other 3 similar commits
>>> which add new bits to insns_flags and insns_flags2 fields of POWER7
>>> and POWER8 CPUs.
>>>
>>> 4e0806110c8b896ceff3490f15a616e8b3165efe Adds PPC_64H to POWER7 and POWER8
>>> dfdd3e43620a6cd4f2be31da5a257c84a16fc000 Adds PPC_64BX to POWER7
>>> b781537560e3b968b6fe1395e3d07bd67f0009ba Adds PPC_CILDST to POWER7 and 
>>> POWER8
>>> 7778a575c7055276afdd01737e9d1029a65f923d Adds PPC2_PM_ISA206 to POWER7 and 
>>> POWER8
>>>
>>> The flag values are expected to remain same for a machine version for
>>> the migration to succeed, but this expectation is broken now. Should
>>> we make the addition of these flags conditional on machine type version ?
>>> But these flags are part of POWER8 CPU definition which is common for
>>> both pseries and upcoming powernv.
>>
>> Can you step me through how the new flags are breaking the migration?
>> It's not immediately obvious to me.
> 
> Here is what I understand. Given below is the pruned vmstate_ppc_cpu
> data structure.
> 
> const VMStateDescription vmstate_ppc_cpu = {
>     .name = "cpu",
>     .fields = (VMStateField[]) {
>         /* Sanity checking */
>         VMSTATE_UINT64_EQUAL(env.insns_flags, PowerPCCPU),
>         VMSTATE_UINT64_EQUAL(env.insns_flags2, PowerPCCPU),
>         VMSTATE_END_OF_LIST()
>     },
> };
> 
> When pseries-2.6 guest is started at the source with QEMU-2.6, insns_flags
> and insns_flags2 will not have PPC_64H, PPC_64BX, PPC_CILDST, PPC2_PM_ISA206
> set. However at the target when pseries-2.6 guest is started with QEMU-2.7,
> these flags will be set. And I believe VMSTATE_UINT64_EQUAL checks above
> will cause migration to fail.

So does this mean that we can not add support for new instructions in 
qemu without breaking migration with older versions ? 

If so, that is really bad, we need to find a way to fix this. Should we
add a 'version' to insns_flags* ? 


C.