[Qemu-devel] [RFC] dump memory when host pci device is used by guest

[Wen Congyang]

Hi, all

'virsh dump' can not work when host pci device is used by guest. We have
discussed this issue here:
http://lists.nongnu.org/archive/html/qemu-devel/2011-10/msg00736.html

We have determined to introduce a new command dump to dump memory. The core
file's format can be elf.

I created a kdump-elf vmcore, and found that it can be used by both crash and 
gdb:

# gdb /usr/lib/debug/lib/modules/2.6.32-71.el6.x86_64/vmlinux /work/core/vmcore
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-48.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from 
/usr/lib/debug/lib/modules/2.6.32-71.el6.x86_64/vmlinux...done.
[New Thread 1691]
[New <main task>]
#0  sysrq_handle_crash (key=99, tty=0x0) at drivers/char/sysrq.c:130
130     drivers/char/sysrq.c: No such file or directory.
        in drivers/char/sysrq.c
(gdb) bt
#0  sysrq_handle_crash (key=99, tty=0x0) at drivers/char/sysrq.c:130
#1  0xffffffff8130d822 in __handle_sysrq (key=99, tty=0x0, check_mask=<value 
optimized out>) at drivers/char/sysrq.c:521
#2  0xffffffff8130d8de in write_sysrq_trigger (file=<value optimized out>, 
buf=<value optimized out>, count=2, ppos=<value optimized out>) at 
drivers/char/sysrq.c:599
#3  0xffffffff811cf31e in proc_reg_write (file=<value optimized out>, 
buf=0x7fdabafea000 <Address 0x7fdabafea000 out of bounds>, count=2, ppos=<value 
optimized out>)
    at fs/proc/inode.c:207
#4  0xffffffff8116c818 in vfs_write (file=0xffff88003c7bb740, 
buf=0x7fdabafea000 <Address 0x7fdabafea000 out of bounds>, count=<value 
optimized out>, pos=0xffff88003767ff48)
    at fs/read_write.c:347
#5  0xffffffff8116d251 in sys_write (fd=<value optimized out>, 
buf=0x7fdabafea000 <Address 0x7fdabafea000 out of bounds>, count=2) at 
fs/read_write.c:399
#6  0xffffffff81013172 in ?? () at arch/x86/kernel/entry_64.S:487
#7  0x0000000000000246 in ?? ()
#8  0x00000000ffffffff in ?? ()
#9  0x00007fdabafde700 in ?? ()
#10 0x000000000000000a in ?? ()
#11 0x0000000000000001 in ?? ()
#12 0x0000000000000002 in ?? ()
#13 0x0000000000000001 in ?? ()
#14 0x00000030f80d4230 in ?? ()
#15 0x0000000000000033 in ?? ()
#16 0x0000000000010206 in ?? ()
#17 0x00007fff8a126470 in ?? ()
#18 0x000000000000002b in ?? ()
#19 0xffff8800374f5000 in ?? ()
#20 0xffff88003c6f9000 in ?? ()
#21 0x0000000000000080 in ?? ()
#22 0xffff880037680080 in ?? ()
#23 0xffffffff00000014 in ?? ()
#24 0x0000000000000000 in ?? ()
(gdb) q
# crash -s /usr/lib/debug/lib/modules/2.6.32-71.el6.x86_64/vmlinux 
/work/core/vmcore 
crash> bt
PID: 1691   TASK: ffff88003711d520  CPU: 0   COMMAND: "bash"
 #0 [ffff88003767fae0] machine_kexec at ffffffff8103695b
 #1 [ffff88003767fb40] crash_kexec at ffffffff810b8f08
 #2 [ffff88003767fc10] oops_end at ffffffff814cbbd0
 #3 [ffff88003767fc40] no_context at ffffffff8104651b
 #4 [ffff88003767fc90] __bad_area_nosemaphore at ffffffff810467a5
 #5 [ffff88003767fce0] bad_area at ffffffff810468ce
 #6 [ffff88003767fd10] do_page_fault at ffffffff814cd740
 #7 [ffff88003767fd60] page_fault at ffffffff814caf45
    [exception RIP: sysrq_handle_crash+22]
    RIP: ffffffff8130d566  RSP: ffff88003767fe18  RFLAGS: 00010096
    RAX: 0000000000000010  RBX: 0000000000000063  RCX: 0000000000000000
    RDX: 0000000000000000  RSI: 0000000000000000  RDI: 0000000000000063
    RBP: ffff88003767fe18   R8: 0000000000000000   R9: ffffffff815106c0
    R10: 0000000000000001  R11: 0000000000000000  R12: 0000000000000000
    R13: ffffffff8179e6c0  R14: 0000000000000286  R15: 0000000000000007
    ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
 #8 [ffff88003767fe20] __handle_sysrq at ffffffff8130d822
 #9 [ffff88003767fe70] write_sysrq_trigger at ffffffff8130d8de
#10 [ffff88003767fea0] proc_reg_write at ffffffff811cf31e
#11 [ffff88003767fef0] vfs_write at ffffffff8116c818
#12 [ffff88003767ff30] sys_write at ffffffff8116d251
#13 [ffff88003767ff80] system_call_fastpath at ffffffff81013172
    RIP: 00000030f80d4230  RSP: 00007fff8a126470  RFLAGS: 00010206
    RAX: 0000000000000001  RBX: ffffffff81013172  RCX: 0000000000000400
    RDX: 0000000000000002  RSI: 00007fdabafea000  RDI: 0000000000000001
    RBP: 00007fdabafea000   R8: 000000000000000a   R9: 00007fdabafde700
    R10: 00000000ffffffff  R11: 0000000000000246  R12: 0000000000000002
    R13: 00000030f8379780  R14: 0000000000000002  R15: 00000030f8379780
    ORIG_RAX: 0000000000000001  CS: 0033  SS: 002b
crash> 

I wrote a sample(not finished). It only can works on x86_64(both host and guest)
I use it to create a core file:
# readelf -h /tmp/vm2.save 
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              CORE (Core file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x0
  Start of program headers:          64 (bytes into file)
  Start of section headers:          0 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         9
  Size of section headers:           0 (bytes)
  Number of section headers:         0
  Section header string table index: 0
# readelf -l /tmp/vm2.save 

Elf file type is CORE (Core file)
Entry point 0x0
There are 9 program headers, starting at offset 64

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  NOTE           0x0000000000000238 0x0000000000000000 0x0000000000000000
                 0x00000000000002c8 0x00000000000002c8         0
  LOAD           0x0000000000000500 0xffffffff81000000 0x0000000001000000
                 0x000000001f000000 0x000000001f000000         0
  LOAD           0x000000001f000500 0x0000000000000000 0x0000000000000000
                 0x0000000001000000 0x0000000001000000         0
  LOAD           0x0000000020000500 0x0000000000000000 0x0000000020000000
                 0x0000000000020000 0x0000000000020000         0
  LOAD           0x0000000020020500 0x0000000000000000 0x0000000020870000
                 0x0000000000010000 0x0000000000010000         0
  LOAD           0x0000000020030500 0x0000000000000000 0x0000000020850000
                 0x0000000000020000 0x0000000000020000         0
  LOAD           0x0000000020050500 0x0000000000000000 0x0000000020840000
                 0x0000000000010000 0x0000000000010000         0
  LOAD           0x0000000020060500 0x0000000000000000 0x0000000020040000
                 0x0000000000800000 0x0000000000800000         0
  LOAD           0x0000000020860500 0x0000000000000000 0x0000000020020000
                 0x0000000000020000 0x0000000000020000         0

I can use crash to anaylze the file, but I can not use gdb to anaylze it.
# gdb /usr/lib/debug/lib/modules/2.6.32-71.el6.x86_64/vmlinux /tmp/vm2.save 
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-48.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from 
/usr/lib/debug/lib/modules/2.6.32-71.el6.x86_64/vmlinux...done.
[New <main task>]
[New <main task>]
#0  0x8103be8b00000000 in ?? ()
(gdb) bt
#0  0x8103be8b00000000 in ?? ()
Cannot access memory at address 0x8170dec800000000
(gdb) q

My first and the most important question is that: Is there necessary to 
continue this work?

The attachment is the sample patch.

Thanks
Wen Congyang

0001-dump-sample.patch
Description: Text Data