[PATCH v5 00/14] GDB script fixes and improvements

[Glenn Washburn]

This series has been substantially rewritten since v4, although a large
minority of the patches haven't changed much. The biggest change is
that the implementation has been converted to Python instead of what was
done in shell script. I have always felt it should be done in Python, but
it seemed daunting to learn the Python-GDB API, so the shell script seemed
the path of least resistance. I decided to give it a second look and was
surprised it wasn't as bad as I thought it would be. Because the python API
is so tightly integrated into GDB, there are things you can do with it that
either aren't possible or which look like ugly hacks when attempting with
the native script.

The other big change is that there is a new --enable-efi-debug option to
configure which enables the printing of the gdb command for loading the
GRUB kernel symbols. This is disallowed when booting with Secure Boot on.
There are two ways the GDB command is printed: (1) in early EFI setup and
(2) on-demand by a user using the gdbinfo command.

There are a couple new features to the GDB script, like a trivial one that
changes the gdb prompt and another that allows for software breakpoints to
be set before the GRUB image is loaded.

This series also incorporates suggestions given for v4 and adds more to the
documentation.

Glenn

Glenn Washburn (14):
  gdb: Fix redirection issue in dump_module_sections
  gdb: Prevent wrapping when writing to .segments.tmp
  gdb: If no modules have been loaded, do not try to load module symbols
  gdb: Move runtime module loading into runtime_load_module
  gdb: Conditionally run GDB script logic for dynamically or statically
    positioned GRUB
  gdb: Only connect to remote target once when first sourced
  gdb: Replace module symbol loading implementation with Python one
  gdb: Add functions to make loading from dynamically positioned targets
    easier
  gdb: Add more support for debugging on EFI platforms
  gdb: Allow running user-defined commands at GRUB start
  gdb: Fix issue with breakpoints defined before the GRUB image is
    loaded
  gdb: Add extra early initialization symbols for i386-pc
  gdb: Modify gdb prompt when running gdb_grub script
  docs: Add debugging chapter to development documentation

 config.h.in                 |   3 +
 configure.ac                |  11 ++
 docs/grub-dev.texi          | 233 ++++++++++++++++++++++++++++++++++++
 grub-core/Makefile.core.def |   5 +-
 grub-core/gdb_grub.in       | 159 +++++++++++++++---------
 grub-core/gdb_helper.py.in  | 173 ++++++++++++++++++++++++++
 grub-core/gmodule.pl.in     |  30 -----
 grub-core/kern/efi/debug.c  |  40 +++++++
 grub-core/kern/efi/efi.c    |   4 +-
 grub-core/kern/efi/init.c   |   7 +-
 include/grub/efi/debug.h    |  43 +++++++
 include/grub/efi/efi.h      |   2 +-
 12 files changed, 620 insertions(+), 90 deletions(-)
 create mode 100644 grub-core/gdb_helper.py.in
 delete mode 100644 grub-core/gmodule.pl.in
 create mode 100644 grub-core/kern/efi/debug.c
 create mode 100644 include/grub/efi/debug.h

Range-diff against v4:
 1:  9f273b8fa5 =  1:  9f273b8fa5 gdb: Fix redirection issue in 
dump_module_sections
 2:  85f68a8369 =  2:  85f68a8369 gdb: Prevent wrapping when writing to 
.segments.tmp
 3:  88b3973cdb =  3:  88b3973cdb gdb: If no modules have been loaded, do not 
try to load module symbols
 4:  c0d7da87a8 !  4:  3037c1da91 gdb: Move runtime module loading into 
runtime_load_module
    @@ Metadata
      ## Commit message ##
         gdb: Move runtime module loading into runtime_load_module
     
    +    By moving this code into a function, it can be run re-utilized while 
gdb is
    +    running, not just when loading the script. This will also be useful in
    +    some following changes which will make a separate script path for 
targets
    +    which statically vs dynamically position GRUB code.
    +
      ## grub-core/gdb_grub.in ##
     @@ grub-core/gdb_grub.in: document load_all_modules
        Load debugging information for all loaded modules.
 5:  4712465374 <  -:  ---------- gdb: Reliably load modules in 
runtime_load_module
 6:  283021b7b9 <  -:  ---------- gdb: Add functions to make loading from 
dynamically positioned targets easier
 7:  8f4b7c3bbd <  -:  ---------- gdb: Remove Perl dependency for GRUB GDB 
script
 8:  055e968779 <  -:  ---------- gdb: If enabled, print line used to load EFI 
kernel symbols when using gdb_grub script
 9:  64eccfc37e =  5:  f6288016f6 gdb: Conditionally run GDB script logic for 
dynamically or statically positioned GRUB
10:  5064458dfd =  6:  da13fbe653 gdb: Only connect to remote target once when 
first sourced
11:  c33e8f57b4 <  -:  ---------- gdb: Allow user defined "onload_<modname>" 
command to be run when module is loaded
 -:  ---------- >  7:  8e6059955a gdb: Replace module symbol loading 
implementation with Python one
 -:  ---------- >  8:  878900d69b gdb: Add functions to make loading from 
dynamically positioned targets easier
 -:  ---------- >  9:  036549604d gdb: Add more support for debugging on EFI 
platforms
12:  f8a26f3a56 ! 10:  0000959b2f gdb: Allow running user-defined commands at 
GRUB start
    @@ Metadata
      ## Commit message ##
         gdb: Allow running user-defined commands at GRUB start
     
    -    A new command, run_on_start, is created which handles some complexities
    -    of the EFI platform when breaking on GRUB start. If GRUB start is 
hooked,
    -    run "onstart" command if it is defned.
    +    A new command, run_on_start, for things to do when just before GRUB 
starts
    +    executing. Currently, this is setting up the loading of module symbols 
as
    +    they are loaded and allowing user-defined script to be run if a command
    +    named "onstart" exists. A thbreak, temporary hardware breakpoint, is 
used
    +    because a software breakpoint would be overwritten when the firmware 
loads
    +    the GRUB image into memory. And it should be temporary in order to 
have as
    +    many of the limited hardware breakpoints available to the user as 
possible.
     
      ## grub-core/gdb_grub.in ##
    -@@ grub-core/gdb_grub.in: end
    +@@ grub-core/gdb_grub.in: source gdb_helper.py
      define dynamic_load_symbols
        dynamic_load_kernel_exec_symbols $arg0
      
    @@ grub-core/gdb_grub.in: document runtime_load_module
     +define run_on_start
     +  # TODO: Add check to see if _start symbol is defined, if not, then
     +  # the symbols have not yet been loaded and this command will not work.
    -+  watch *_start
    -+  set $break_efi_start_bpnum = $bpnum
    ++  thbreak _start
     +  commands
     +          silent
    -+          delete $break_efi_start_bpnum
    -+          break _start
    -+          commands
    -+                  silent
    -+                  delete $break_efi_start_bpnum
    -+                  set $onstart_name = "onstart"
    -+                  is_user_command $onstart_name
    -+                  if $ret
    -+                          onstart
    -+                  end
    -+                  continue
    ++
    ++          runtime_load_module
    ++
    ++          if $is_user_command("onstart")
    ++                  onstart
     +          end
    -+          set $break_efi_start_bpnum = $bpnum
     +          continue
     +  end
     +end
     +document run_on_start
     +  On some targets, such as x86_64-efi, even if you know where the
    -+  firmware will load the grub image, you can not simply set a break
    ++  firmware will load the GRUB image, you can not simply set a break
     +  point before the image is loaded because loading the image
     +  overwrites the break point in memory. So setup a hardware watch
     +  point, which does not have that problem, and if that gets triggered,
 -:  ---------- > 11:  ac9f52b1d9 gdb: Fix issue with breakpoints defined 
before the GRUB image is loaded
13:  fbd217a89c ! 12:  eac4405ffb gdb: Add extra early initialization symbols 
for i386-pc
    @@ Commit message
         gdb: Add extra early initialization symbols for i386-pc
     
         Add symbols for boot.image, disk.image, and lzma_decompress.image if 
the
    -    target is i386-pc.
    +    target is i386-pc. This is only done for i386-pc because that is the 
only
    +    target that uses the images. By loading the symbols for these images,
    +    these images can be more easily debugged by allowing the setting of 
break-
    +    points in that code and to see easily get the value of data symbols.
     
      ## grub-core/gdb_grub.in ##
     @@ grub-core/gdb_grub.in: set confirm off
14:  973f24a485 <  -:  ---------- gdb: Add ability to turn on shell tracing for 
gdb helper script
 -:  ---------- > 13:  e58715e227 gdb: Modify gdb prompt when running gdb_grub 
script
15:  d6c6947762 ! 14:  1979dc664e docs: Add debugging chapter to development 
documentation
    @@ Metadata
      ## Commit message ##
         docs: Add debugging chapter to development documentation
     
    +    Debugging GRUB can be tricky and require arcane knowledge. This will
    +    help those unfamiliar with the process to get started debugging GRUB
    +    with less effort.
    +
      ## docs/grub-dev.texi ##
     @@ docs/grub-dev.texi: This edition documents version @value{VERSION}.
      * Contributing Changes::
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +does not have the debugging facilities normally provided by an operating
     +system. This chapter aims to provide useful information on some ways to
     +debug GRUB2 for some architectures. It by no means intends to be 
exhaustive.
    -+The focus will be one X86_64 and i386 architectures. Luckily for some 
issues
    ++The focus will be one x86_64 and i386 architectures. Luckily for some 
issues
     +virtual machines have made the ability to debug GRUB2 much easier, and 
this
     +chapter will focus debugging via the QEMU virtual machine. We will not be
     +going over debugging of the userland tools (eg. grub-install), there are
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +
     +@example
     +qemu-system-i386 -drive file=disk.img,format=raw \
    -+    -device virtio-scsi-pci,id=scsi0,num_queues=4 -S -s
    ++    -device virtio-scsi-pci,id=scsi0 -S -s
     +@end example
     +
     +This will start a QEMU instance booting from @file{disk.img}. It will 
pause
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +@section x86_64-efi
     +
     +Using GDB to debug GRUB2 for the x86_64-efi target has some similarities 
with
    -+the i386-pc target. Please read be familiar with the @ref{x86_64-efi} 
section
    ++the i386-pc target. Please read be familiar with the @ref{i386-pc} section
     +when reading this one. Extra care must be used to run QEMU such that it 
boots
     +a UEFI firmware. This usually involves either using the @samp{-bios} 
option
     +with a UEFI firmware blob (eg. @file{OVMF.fd}) or loading the firmware via
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +Like all EFI implementations, on x86_64-efi the (U)EFI firmware that loads
     +the GRUB2 EFI application determines at runtime where the application will
     +be loaded. This means that we do not know where to tell GDB to load the
    -+symbols for the GRUB2 core until the (U)EFI firmware determines it. There
    ++symbols for the GRUB2 core until the (U)EFI firmware determines it. There 
are
     +two good ways of figuring this out when running in QEMU: use a @ref{OVMF 
debug log,
     +debug build of OVMF} and check the debug log or have GRUB2 say where it is
     +loaded when it starts. Neither of these are ideal because they both
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +between QEMU runs. There are exceptions to this, namely that different
     +GRUB2 EFI Applications can be run at different addresses. Also, its been
     +observed that after running the EFI application for the first time, the
    -+second run will many times have a different load address, but subsequent
    ++second run will some times have a different load address, but subsequent
     +runs of the same EFI application will have the same load address as the
    -+second run. This predictability allows us to asume the load address on
    -+subsequent runs and thus load the symbols before GRUB2 starts. The 
following
    -+command illustrates this, assuming that QEMU is running and waiting for
    -+a debugger connection and the current working directory is where
    -+@file{gdb_grub} resides:
    ++second run. And its a near certainty that if the GRUB EFI binary has 
changed,
    ++eg. been recompiled, the load address will also be different.
    ++
    ++This ability to predict what the load address will be allows one to assume
    ++the load address on subsequent runs and thus load the symbols before GRUB2
    ++starts. The following command illustrates this, assuming that QEMU is
    ++running and waiting for a debugger connection and the current working
    ++directory is where @file{gdb_grub} resides:
     +
     +@example
    -+gdb -x gdb_grub -ex 'dynamic_load_symbols @var{load address}'
    ++gdb -x gdb_grub -ex 'dynamic_load_symbols @var{address of .text section}'
     +@end example
     +
     +If you load the symbols in this manner and, after continuing execution, do
     +not see output showing the loading of modules symbol, then its very likely
     +that the load address was incorrect.
     +
    ++Another thing to be aware of is how the loading of the GRUB image by the
    ++firmware affects previously set software breakpoints. On x86 platforms,
    ++software breakpoints are implemented by GDB by writing a special processor
    ++instruction at the location of the desired breakpoint. This special 
instruction
    ++when executed will stop the program execution and hand control to the
    ++debugger, GDB. GDB will first saves the instruction bytes that will be
    ++overwritten at the breakpoint, and will put them back when the breakpoint
    ++is hit. If GRUB is being run for the first time in QEMU, the firmware will
    ++be loading the GRUB image into memory where every byte is already set to 
0.
    ++This means that if a breakpoint is set before GRUB is loaded, GDB will 
save
    ++the 0-byte(s) where the the special instruction will go. Then when the 
firmware
    ++loads the GRUB image and because it is unaware of the debugger, it will
    ++write the GRUB image to memory, overwriting anything that was there 
previously,
    ++notably in this case the instruction that implements the software 
breakpoint.
    ++This will be confusing for the person using GDB because GDB will show the
    ++breakpoint as set, but the brekapoint will never be hit. Furthermore, GDB
    ++then become confused, such that even deleting an recreating the breakpoint
    ++will not create usable breakpoints. The @file{gdb_grub} script takes care 
of
    ++this by saving the breakpoints just before they are overwritten, and then
    ++restores them at the start of GRUB execution. So breakpoints for GRUB can 
be
    ++set before GRUB is loaded, but be mindful of this effect if you are 
confused
    ++as to why breakpoints are not getting hit.
    ++
    ++Also note, that hardware breakpoints do not suffer this problem. They are
    ++implemented by having the breakpoint address in special debug registers on
    ++the CPU. So they can always be set freely without regard to whether GRUB 
has
    ++been loaded or not. The reason that hardware breakpoints aren't always 
used
    ++is because there are a limited number of them, usually around 4 on various
    ++CPUs, and specifically exactly 4 for x86 CPUs. The @file{gdb_grub} script
    ++goes out of its way to not use hardware breakpoints internally and when
    ++needed use them as short a time as possible, thus allowing the user to 
have a
    ++maximal number at their disposal.
    ++
     +
     +@node OVMF debug log
     +@subsection OVMF debug log
    @@ docs/grub-dev.texi: cp minilzo-2.10/*.[hc] grub-core/lib/minilzo
     +@example
     +qemu-system-x86_64 -bios /path/to/debug/OVMF.fd \
     +    -drive file=disk.img,format=raw \
    -+    -device virtio-scsi-pci,id=scsi0,num_queues=4 \
    ++    -device virtio-scsi-pci,id=scsi0 \
     +    -debugcon file:debug.log -global isa-debugcon.iobase=0x402
     +@end example
     +
     +If GRUB2 was started by the (U)EFI firmware, then in the @file{debug.log}
     +file one of the last lines should be a log message like:
    -+@code{Loading driver at 0x00006AEE000 EntryPoint=0x00006AEE756}. This
    -+means that the GRUB2 EFI application was loaded at @code{0x00006AEE000} 
and
    -+its .text section is at @code{0x00006AEE756}.
    ++@samp{Loading driver at 0x00006AEE000 EntryPoint=0x00006AEE756}. This
    ++means that the GRUB2 EFI application was loaded at @samp{0x00006AEE000} 
and
    ++its .text section is at @samp{0x00006AEE756}.
     +
     +@node Build GRUB2 to print out the load address
     +@subsection Build GRUB2 to print out the load address
     +
     +GRUB2 can be specially built to output the address of its .text section in
    -+memory by defining @code{PRINT_GDB_SYM_LOAD_CMD} to @code{1} in 
@file{config.h.in}
    -+before running @command{configure}. The benefit of this method is that it
    -+will work on non-virtualized hardware where the (U)EFI firmware may not
    -+be modifiable.
    ++memory by using the @samp{--enable-efi-debug} configure option. The 
benefit
    ++of this method is that it will work on non-virtualized hardware where the
    ++(U)EFI firmware may not be modifiable. This option has no effect when 
booting
    ++with Secure Boot enabled. Otherwise, GRUB will print the gdb command to 
use
    ++very early in GRUB startup. The text quickly gets overwritten, perhaps 
even
    ++too quickly to see when booting with a physical monitor as the only output
    ++source. For this reason, a command named "gdbinfo" is enabled which will
    ++print the same output. So the user can get this info at anytime. If GRUB 
is
    ++crashing before the commandline can be reached, this will be of no help
    ++unfortunately.
     +
      @node Porting
      @chapter Porting
-- 
2.34.1