[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] qemu 2.0, deadlock in block-commit
From: |
Stefan Hajnoczi |
Subject: |
Re: [Qemu-devel] qemu 2.0, deadlock in block-commit |
Date: |
Fri, 23 May 2014 14:58:47 +0200 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On Thu, May 22, 2014 at 10:49:18PM +0200, Marcin Gibuła wrote:
> This is backtrace of qemu process:
>
> (gdb) thread apply all backtrace
[...] a bunch of rbd threads, vnc worker thread, QEMU worker threads
> Thread 1 (Thread 0x7f699bfcd900 (LWP 13647)):
> #0 0x00007f6998020286 in ppoll () from /lib64/libc.so.6
> #1 0x00007f699c1f3d9b in ppoll (__ss=0x0, __timeout=0x0, __nfds=<optimized
> out>, __fds=<optimized out>) at /usr/include/bits/poll2.h:77
> #2 qemu_poll_ns (fds=<optimized out>, nfds=<optimized out>,
> timeout=<optimized out>) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/qemu-timer.c:311
> #3 0x00007f699c0877e0 in aio_poll (ctx=0x7f699e4c9c00,
> address@hidden) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/aio-posix.c:221
> #4 0x00007f699c095c0a in bdrv_drain_all () at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/block.c:1805
QEMU is waiting for all block I/O requests to complete. I wonder if
there is some weird interaction with rbd here, which is why this never
completes.
In gdb you can iterate over bdrv_states to inspect the open
BlockDriverState structs. Each BDS struct has a tracked_requests list
and bdrv_drain_all() is waiting for this pending requests list to become
empty.
If you see a pending request on a RADOS block device (rbd) then it would
be good to dig deeper into QEMU's block/rbd.c driver to see why it's not
completing that request.
Are you using qcow2 on top of rbd?
> #5 0x00007f699c09c87e in bdrv_close (address@hidden) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/block.c:1695
> #6 0x00007f699c09c5fa in bdrv_delete (bs=0x7f699f0bc520) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/block.c:1978
> #7 bdrv_unref (bs=0x7f699f0bc520) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/block.c:5198
> #8 0x00007f699c09c812 in bdrv_drop_intermediate
> (address@hidden, address@hidden,
> address@hidden) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/block.c:2567
> #9 0x00007f699c0a1963 in commit_run (opaque=0x7f699f17dcc0) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/block/commit.c:144
> #10 0x00007f699c0e0dca in coroutine_trampoline (i0=<optimized out>,
> i1=<optimized out>) at
> /var/tmp/portage/app-emulation/qemu-2.0.0_rc2/work/qemu-2.0.0-rc2/coroutine-ucontext.c:118
> #11 0x00007f6997f859f0 in ?? () from /lib64/libc.so.6
> #12 0x00007fffdbe06750 in ?? ()
> #13 0x0000000000000000 in ?? ()
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, (continued)
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Paolo Bonzini, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Kevin Wolf, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Eric Blake, 2014/05/22
Re: [Qemu-devel] qemu 2.0, deadlock in block-commit,
Stefan Hajnoczi <=
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/23
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Stefan Hajnoczi, 2014/05/26
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/26
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Stefan Hajnoczi, 2014/05/28
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/28
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Stefan Hajnoczi, 2014/05/28
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/28
- Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/29
Re: [Qemu-devel] qemu 2.0, deadlock in block-commit, Marcin Gibuła, 2014/05/30