and found that the only possible is the emulation of 'cpuid' instruct in
kvm/qemu has something wrong.
But since we can’t reproduce this problem, we are not quite sure.
Is there any possible that the cupid emulation in kvm/qemu has some bug ?
Can you explain the relationship to the cpuid emulation? What do the
traces say about vcpus 1 and 7?
OK, we searched the VM's kernel codes with the 'Stuck' message, and it is
located in
do_boot_cpu(). It's in BSP context, the call process is:
BSP executes start_kernel() -> smp_init() -> smp_boot_cpus() -> do_boot_cpu()
-> wakeup_secondary_via_INIT() to trigger APs.
It will wait 5s for APs to startup, if some AP not startup normally, it will
print 'CPU%d Stuck' or 'CPU%d: Not responding'.
If it prints 'Stuck', it means the AP has received the SIPI interrupt and
begins to execute the code
'ENTRY(trampoline_data)' (trampoline_64.S) , but be stuck in some places before
smp_callin()(smpboot.c).
The follow is the starup process of BSP and AP.
BSP:
start_kernel()
->smp_init()
->smp_boot_cpus()
->do_boot_cpu()
->start_ip = trampoline_address(); //set the address that AP will
go to execute
->wakeup_secondary_cpu_via_init(); // kick the secondary CPU
->for (timeout = 0; timeout < 50000; timeout++)
if (cpumask_test_cpu(cpu, cpu_callin_mask)) break;// check if
AP startup or not
APs:
ENTRY(trampoline_data) (trampoline_64.S)
->ENTRY(secondary_startup_64) (head_64.S)
->start_secondary() (smpboot.c)
->cpu_init();
->smp_callin();
->cpumask_set_cpu(cpuid, cpu_callin_mask); ->Note: if AP
comes here, the BSP will not prints the error message.
From above call process, we can be sure that, the AP has been stuck between
trampoline_data and the cpumask_set_cpu() in
smp_callin(), we look through these codes path carefully, and only found a
'hlt' instruct that could block the process.
It is located in trampoline_data():
ENTRY(trampoline_data)
...
call verify_cpu # Verify the cpu supports long mode
testl %eax, %eax # Check for return code
jnz no_longmode
...
no_longmode:
hlt
jmp no_longmode
For the process verify_cpu(),
we can only find the 'cpuid' sensitive instruct that could lead VM exit from
No-root mode.
This is why we doubt if cpuid emulation is wrong in KVM/QEMU that leading to
the fail in verify_cpu.
From the message in VM, we know vcpu1 and vcpu7 is something wrong.
[ 5.060042] CPU1: Stuck ??
[ 10.170815] CPU7: Stuck ??
[ 10.171648] Brought up 6 CPUs
Besides, the follow is the cpus message got from host.
80FF72F5-FF6D-E411-A8C8-000000821800:/home/fsp/hrg # virsh qemu-monitor-command
instance-0000000
* CPU #0: pc=0x00007f64160c683d thread_id=68570
CPU #1: pc=0xffffffff810301f1 (halted) thread_id=68573
CPU #2: pc=0xffffffff810301e2 (halted) thread_id=68575
CPU #3: pc=0xffffffff810301e2 (halted) thread_id=68576
CPU #4: pc=0xffffffff810301e2 (halted) thread_id=68577
CPU #5: pc=0xffffffff810301e2 (halted) thread_id=68578
CPU #6: pc=0xffffffff810301e2 (halted) thread_id=68583
CPU #7: pc=0xffffffff810301f1 (halted) thread_id=68584
Oh, i also forgot to mention in the above message that, we have bond each vCPU
to different physical CPU in
host.
Thanks,
zhanghailiang
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to address@hidden
More majordomo info at http://vger.kernel.org/majordomo-info.html