Re: Kernel patch cases qemu live migration failed.

[Andrew Jones]

On Thu, Oct 15, 2020 at 02:52:34PM +0100, Marc Zyngier wrote:
> On 2020-10-15 14:35, Andrew Jones wrote:
> > On Thu, Oct 15, 2020 at 12:26:10PM +0100, Marc Zyngier wrote:
> > > Hi
> > > 
> > > Please don't use my arm.com address anymore, nobody reads it...
> > > 
> > > On 2020-10-15 05:06, 张东旭 wrote:
> > > > I'm so sorry for disturbing you.
> > > >
> > > > When I apply this kernel patch：KVM: arm64/sve: System register
> > > > context switch and access support
> > > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/arm64/kvm/sys_regs.c?id=73433762fcaeb9d59e84d299021c6b15466c96dd
> > > 
> > > This patch was only introduced in 5.2, and wasn't supposed to be
> > > backported to anything else.
> > 
> > All upstream patches are backport candidates. Welcome to RHEL :-)
> 
> Great! RHEL gets to keep the pieces! ;-)
> 
> > 
> > > 
> > > > qemu live migration will failed with messages：
> > > >               ​qemu-kvm: Invalid value 233 expecting positive value
> > > > <= 232
> > > >               qemu-kvm: Failed to load cpu:cpreg_vmstate_array_len
> > > > New version kernel  exclude ID_AA64ZFR0_EL1 register when host not
> > > > support SVE feature,
> > > > so qemu ioctl kvm(KVM_GET_REG_LIST) will not contain ID_AA64ZFR0_EL1
> > > > register.
> > > >
> > > > I'm using CentOS Linux kernel, old kernel version：4.18.0-80.11.el8
> > > > (migration source)
> > > >       new kernel version：4.18.0-147.5.el8 (migration destination)
> > > >
> > > > CentOS linux kernel version 4.18.0-111.el8 applied this patch. So
> > > > 4.18.0-147.5.el8 also applied this patch.
> > > >
> > > > Migration source and destination hosts have the same hardware, just
> > > > kernel version is different,
> > > > and the hardware on either side of the migration not support SVE.
> > > >
> > > > Is there some good suggestions，which can make sure old version
> > > > kernel live migration to new version kernel with qemu?
> > > 
> > > I['m afraid you should take this with your distribution of
> > > choice, unless you can reproduce the problem with mainline kernels.
> > 
> > I'll bet this reproduces when migrating from a mainline pre SVE register
> > filtering kernel to a mainline that does the filtering. I don't have
> > time to test that though.
> 
> Probably. I'm not really sure where to find SVE HW though, other
> than emulation...
>

The reporter states neither the source nor destination hardware supports
SVE. My guess is that what's happening is the reserved ID register
ID_UNALLOCATED(4,4) was showing up in the KVM_GET_REG_LIST count on
the old kernel, but the new kernel filters it out. Maybe it is a
bug to filter it out of the count, as it's a reserved ID register and
I suppose the other reserved ID registers are still showing up?

Thanks,
drew