Re: [Qemu-block] forward of qemu bug regarding random qemu hangs at shutdown

[Marc Hartmayer]

On Fri, Sep 07, 2018 at 06:21 PM +0200, Marc Hartmayer <address@hidden> wrote:
> On Fri, Sep 07, 2018 at 11:34 AM +0200, Kevin Wolf <address@hidden> wrote:
>> Am 06.09.2018 um 21:29 hat Christian Borntraeger geschrieben:

[…snip…]

>>
>> I think we need more information there. Can you set a breakpoint on
>> bdrv_drain_all_begin() to see where it's even called? When I start a
>> qemu instance without a block device, the first time this is called is
>> during shutdown after the mainloop (i.e. after the place where you're
>> seeing a hang).
>
> I can try that.

Okay, I’ve instrumented that function using SystemTap.

The first (and actually the only) call of bdrv_drain_all_begin of a
hanging QEMU is

161.311 bdrv_drain_all_begin 19105
 0x559bdc933045 : _nocheck__trace_bdrv_drain_all_begin+0x4/0x8 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc933052 : trace_bdrv_drain_all_begin+0x9/0xc 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc93472d : bdrv_drain_all_begin+0x16/0x21e 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc9349e5 : bdrv_drain_all+0x9/0x11 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc49beba : do_vm_stop+0x5a/0x6c 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc49db56 : vm_stop+0x3e/0x40 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc61a77a : main_loop_should_exit+0xae/0x191 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc61a872 : main_loop+0x15/0x1f 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x559bdc6221ac : main+0x3f1c/0x3f7b 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x7f41160a688a [/usr/lib64/libc-2.25.so+0x2088a/0x3d5000]


The calls of bdrv_drain_all_begin of a non-hanging QEMU are:

160.651 bdrv_drain_all_begin 19044
 0x55f558a0d045 : _nocheck__trace_bdrv_drain_all_begin+0x4/0x8 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0d052 : trace_bdrv_drain_all_begin+0x9/0xc 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0e72d : bdrv_drain_all_begin+0x16/0x21e 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0e9e5 : bdrv_drain_all+0x9/0x11 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558575eba : do_vm_stop+0x5a/0x6c 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558577b56 : vm_stop+0x3e/0x40 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f5586f477a : main_loop_should_exit+0xae/0x191 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f5586f4872 : main_loop+0x15/0x1f 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f5586fc1ac : main+0x3f1c/0x3f7b 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x7f48c929f88a [/usr/lib64/libc-2.25.so+0x2088a/0x3d5000]
160.652 bdrv_drain_all_begin 19044
 0x55f558a0d045 : _nocheck__trace_bdrv_drain_all_begin+0x4/0x8 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0d052 : trace_bdrv_drain_all_begin+0x9/0xc 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0e72d : bdrv_drain_all_begin+0x16/0x21e 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0e9e5 : bdrv_drain_all+0x9/0x11 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558575eba : do_vm_stop+0x5a/0x6c 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558575edf : vm_shutdown+0x13/0x15 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f5586fc1b6 : main+0x3f26/0x3f7b 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x7f48c929f88a [/usr/lib64/libc-2.25.so+0x2088a/0x3d5000]
160.652 bdrv_drain_all_begin 19044
 0x55f558a0d045 : _nocheck__trace_bdrv_drain_all_begin+0x4/0x8 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0d052 : trace_bdrv_drain_all_begin+0x9/0xc 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0e72d : bdrv_drain_all_begin+0x16/0x21e 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f558a0e9e5 : bdrv_drain_all+0x9/0x11 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f55899e4a4 : bdrv_close_all+0x3c/0x74 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x55f5586fc1c0 : main+0x3f30/0x3f7b 
[/usr/local/qemu/master/bin/qemu-system-x86_64]
 0x7f48c929f88a [/usr/lib64/libc-2.25.so+0x2088a/0x3d5000]

Hope this helps.

[…snip]

--
Kind regards / Beste Grüße
   Marc Hartmayer

IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294