Re: [Qemu-devel] Qemu 2.6 => Qemu 2.7 migration: error while loading state for instance 0x0 of device '0000:00:03.0/virtio-balloon'

[Ladi Prosek]

On Tue, Nov 15, 2016 at 12:19 PM, Stefan Priebe - Profihost AG
<address@hidden> wrote:
> Am 15.11.2016 um 12:07 schrieb Ladi Prosek:
>> Hi,
>>
>> On Tue, Nov 15, 2016 at 11:37 AM, Stefan Priebe - Profihost AG
>> <address@hidden> wrote:
>>> Hello,
>>>
>>> Am 15.11.2016 um 11:30 schrieb Dr. David Alan Gilbert:
>>>> * Stefan Priebe - Profihost AG (address@hidden) wrote:
>>>>> Hello,
>>>>>
>>>>> today i did a first live migration from Qemu 2.6.2 to Qemu 2.7.0. The VM
>>>>> is running windows and virtio-balloon and with machine type pc-i440fx-2.5.
>>>>>
>>>>> The output of the target qemu process was:
>>>>> kvm_apic_post_load: Yeh
>>>>> kvm_apic_post_load: Yeh
>>>>> kvm_apic_post_load: Yeh
>>>>> kvm_apic_post_load: Yeh
>>>>> kvm: VQ 2 size 0x80 < last_avail_idx 0x1 - used_idx 0x4
>>>>> kvm: error while loading state for instance 0x0 of device
>>>>> '0000:00:03.0/virtio-balloon'
>>>>> kvm: load of migration failed: Operation not permitted
>>>>
>>>> Yes that's a known bug; only seems to affect windows guests, and I believe
>>>> doesn't even need to cross versions.
>>>>
>>>> There's a bunch of fixes that Stefan applied to virtio code
>>>> that I think fix this; I see that he cc'd qemu-stable.
>>>> I think it's 4b7f91ed, but I'm not sure if there are others needed.
>>>
>>> thanks for pointing to that commit.
>>>
>>> Stefan can you tell me whether it's enough to cherry-pick 4b7f91ed into
>>> 2.7.0 ?
>>
>> I don't believe that 4b7f91ed will help here (no device reset on
>> migration). We've seen this error with QEMU running without:
>>
>>   commit 4eae2a657d1ff5ada56eb9b4966eae0eff333b0b
>>   Author: Ladi Prosek <address@hidden>
>>   Date:   Tue Mar 1 12:14:03 2016 +0100
>>
>>       balloon: fix segfault and harden the stats queue
>>
>>
>> Is it possible that the VM has run on such a QEMU, then was
>> live-migrated to 2.6.2, and then to 2.7.0?
>
> Hi,
>
> yes, it was started under Qemu 2.5.0. Was then migrated to 2.6.2 and
> then to 2.7.0.

Thanks. Then I'm afraid there's currently no QEMU fix for the error
you're getting. The virtqueue got into an inconsistent state in 2.5.0
and it carried over to 2.7.0 where the check triggered. As far as I
can tell, the only thing we could do to fix this scenario in 2.7.0
would be to relax the check. It runs before any of the virtio-balloon
code runs.

What makes this specific to Windows guests is that the Windows balloon
driver sends memory stats only if the balloon service (blnsvr.exe) is
running. Older QEMU pulls memory stats unconditionally even if there's
nothing sending them on the guest side.

One possible workaround would be to make sure that blnsvr.exe is
running before the migration. The binary is part of the virtio-win
ISO. I don't think that it guarantees getting around the check but it
may be worth trying.

> Greets,
> Stefan
>
>>
>> Thanks,
>> Ladi
>>
>>> Greets,
>>> Stefan
>>>
>>>>
>>>> Dave
>>>>>
>>>>> Greets,
>>>>> Stefan
>>>>>
>>>> --
>>>> Dr. David Alan Gilbert / address@hidden / Manchester, UK
>>>>
>>>