Re: [Qemu-devel] [PATCH] kvmclock: Ensure time in migration never goes backward

[Marcin Gibuła]

On 06.05.2014 01:31, Marcelo Tosatti wrote:
> Marcin,
>
> Can you provide detailed instructions on how to reproduce the problem?

Some time ago I found that some virtual machines hang after migration. 
Problem seemed to be linux specific (windows guests never hanged) and 
somehow releated to kvm-clock. After seeking solution on qemu-devel I 
got mail from a guy who also encountered same problem and had VM image 
saved. We were both able to reproduce the same hang, using different 
host kernels and configs.

This VM is clean install of Centos 6.5 but I'm currently not permitted 
to distribute its saved image.

How to reproduce:
1. Save VM to file,
2. Restore VM from file,
3. Connect to VNC,
4. If console is responsive, kill VM and goto 2.

The facts we established:
- VM _sometime_ hangs on restore. On my system it was about 1 on 20 
restores,
- I don't know if it's a bug with saving VM state or with restoring it. 
The random chance suggests the latter,
- VM hangs for some undeterminated time and then resumes with date in 
far future (i.e. 500 years ahead, but I've seen larger values as well),
- only system time on VM is corrupted, hwclock returns correct value,
- the frequency of this bug somehow depends on host kernel config. I.e. 
changing HZ to 250 makes it more likely,
- the VM is unresponsive to anything than NMI due to spinning with 
interrupts disabled in __run_timers function in kernel/timer.c:

while (time_after_eq(jiffies, base->timer_jiffies)) {
  ...
}

(though my disassembly skills are limited so take it with a grain of salt)

- running vm kernel with nokvmclock option fixes the problem,
- VM responds to NMI with usual message with constant timestamp (as if 
time is frozen in VM),
- I've seen this with all types of linux guests with different guest 
kernels - debian, ubuntu, centos,
- I've seen this with different host kernels as well, the earliest I 
tested was 3.8, the newest 3.12,
- I've seen this on both Intel and AMD hosts.

I'm attaching my host .config.

--
mg

kernel-config
Description: Text document