On Mon, Dec 07, 2015 at 07:44:24PM -0500, Neil McGill wrote:
Has anyone seen anything like this ? all 4 qemu threads are stuck on a
pthread condition
This is incorrect. Thread 4 is blocked in poll(2).
QEMU emulator version 2.2.0 (Debian 1:2.2+dfsg-5expubuntu9.3~cloud0), Copyright
(c) 2003-2008 Fabrice Bellard
I'll try and get the qemu source built on this machine to debug and get
some useful symbols out of gdb, but just wondering if there is a known
issue like this on:
Linux virl 3.19.0-33-generic #38~14.04.1-Ubuntu SMP Fri Nov 6 18:17:28 UTC 2015
x86_64 x86_64 x86_64 GNU/Linux
tx
neil
(gdb) where
#0 __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1 0x00007f7a17524657 in _L_lock_909 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#2 0x00007f7a17524480 in __GI___pthread_mutex_lock (mutex=0x7f7a2150f9a0) at
../nptl/pthread_mutex_lock.c:79
#3 0x00007f7a21036909 in ?? ()
#4 0x00007f7a20fc1fdb in ?? ()
#5 0x00007f7a20d153fe in ?? ()
#6 0x00007f7a17176ec5 in __libc_start_main (main=0x7f7a20d13eb0, argc=94,
argv=0x7ffcb0b0b948, init=<optimized out>,
fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffcb0b0b938)
at libc-start.c:287
#7 0x00007f7a20d1b96c in ?? ()
This is the QEMU main loop. It's probably trying to lock the global
mutex.
(gdb) thread 2
[Switching to thread 2 (Thread 0x7f793e3ff700 (LWP 65315))]
#0 pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
185 ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S: No such
file or directory.
(gdb) where
#0 pthread_cond_wait@@GLIBC_2.3.2 () at
../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
#1 0x00007f7a21036b29 in ?? ()
#2 0x00007f7a20fb10c3 in ?? ()
#3 0x00007f7a20fb14c0 in ?? ()
#4 0x00007f7a17522182 in start_thread (arg=0x7f793e3ff700) at
pthread_create.c:312
#5 0x00007f7a1724f47d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb) thread 3
[Switching to thread 3 (Thread 0x7f7a0cbf8700 (LWP 65311))]
#0 __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
135 ../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S: No such file or
directory.
(gdb) where
#0 __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/x86_64/lowlevellock.S:135
#1 0x00007f7a17524657 in _L_lock_909 () from
/lib/x86_64-linux-gnu/libpthread.so.0
#2 0x00007f7a17524480 in __GI___pthread_mutex_lock (mutex=0x7f7a2150f9a0) at
../nptl/pthread_mutex_lock.c:79
#3 0x00007f7a21036909 in ?? ()
#4 0x00007f7a20d57d3c in ?? ()
#5 0x00007f7a20d42db2 in ?? ()
#6 0x00007f7a17522182 in start_thread (arg=0x7f7a0cbf8700) at
pthread_create.c:312
#7 0x00007f7a1724f47d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
This could be a vcpu thread. It's also trying to lock the global mutex.
(gdb) thread 4
[Switching to thread 4 (Thread 0x7f7a0d3f9700 (LWP 65310))]
#0 0x00007f7a172421ef in __GI_ppoll (fds=0x7f7a234db150, nfds=1,
timeout=<optimized out>, sigmask=0x0)
at ../sysdeps/unix/sysv/linux/ppoll.c:56
56 ../sysdeps/unix/sysv/linux/ppoll.c: No such file or directory.
(gdb) where
#0 0x00007f7a172421ef in __GI_ppoll (fds=0x7f7a234db150, nfds=1,
timeout=<optimized out>, sigmask=0x0)
at ../sysdeps/unix/sysv/linux/ppoll.c:56
#1 0x00007f7a20fc2b2b in ?? ()
#2 0x00007f7a20fc3d04 in ?? ()
#3 0x00007f7a20fb5e5f in ?? ()
#4 0x00007f7a20fb78f4 in ?? ()
#5 0x00007f7a20ef1732 in ?? ()
#6 0x00007f7a20d58e81 in ?? ()
#7 0x00007f7a20d5e947 in ?? ()
#8 0x00007f7a20d1f933 in ?? ()
#9 0x00007f7a20d57fa1 in ?? ()
#10 0x00007f7a20d42db2 in ?? ()
#11 0x00007f7a17522182 in start_thread (arg=0x7f7a0d3f9700) at
pthread_create.c:312
#12 0x00007f7a1724f47d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
(gdb)
This is the interesting thread. It is blocked in poll(2) waiting for
activity on just 1 file descriptor. You could check fds[0] and then
look at ls -l /proc/fd/<n> to see what the file is being monitored.
My guess is thread 4 holds the global mutex and is therefore blocking
all the other threads.
Stefan