Re: [Qemu-devel] [RFC PATCH v8 00/21] Deterministic replay core

[Paolo Bonzini]

On 28/01/2015 12:45, Pavel Dovgaluk wrote:
> Ping?

Reviewed 13 patches out of 21.  Made some comments, but overall I'm
really pleased.  Thanks for persisting!

Will continue tomorrow.

Paolo

> Pavel Dovgalyuk
> 
>> -----Original Message-----
>> From: Pavel Dovgalyuk [mailto:address@hidden
>> Sent: Thursday, January 22, 2015 11:52 AM
>> To: address@hidden
>> Cc: address@hidden; address@hidden; address@hidden;
>> address@hidden; address@hidden; address@hidden; address@hidden;
>> address@hidden; address@hidden; address@hidden; address@hidden;
>> address@hidden
>> Subject: [RFC PATCH v8 00/21] Deterministic replay core
>>
>> This set of patches is related to the reverse execution and deterministic
>> replay of qemu execution  This implementation of deterministic replay can
>> be used for deterministic debugging of guest code through gdb remote
>> interface.
>>
>> These patches include only core function of the replay,
>> excluding the support for replaying serial, audio, network, and USB devices'
>> operations. Reverse debugging and monitor commands were also excluded to
>> be submitted later as separate patches.
>>
>> Execution recording writes non-deterministic events log, which can be later
>> used for replaying the execution anywhere and for unlimited number of times.
>> It also supports checkpointing for faster rewinding during reverse debugging.
>> Execution replaying reads the log and replays all non-deterministic events
>> including external input, hardware clocks, and interrupts.
>>
>> Deterministic replay has the following features:
>>  * Deterministically replays whole system execution and all contents of the 
>> memory,
>>    state of the hadrware devices, clocks, and screen of the VM.
>>  * Writes execution log into the file for latter replaying for multiple times
>>    on different machines.
>>  * Supports i386, x86_64, and ARM hardware platforms.
>>  * Performs deterministic replay of all operations with keyboard and mouse
>>    input devices.
>>  * Supports auto-checkpointing for convenient reverse debugging.
>>
>> Usage of the record/replay:
>>  * First, record the execution, by adding the following string to the 
>> command line:
>>    '-record fname=replay.bin -icount 7 -net none'. Block devices' images are 
>> not
>>    actually changed in the recording mode, because all of the changes are
>>    written to the temporary overlay file.
>>  * Then you can replay it for the multiple times by using another command
>>    line option: '-replay fname=replay.bin -icount 7 -net none'
>>  * '-net none' option should also be specified if network replay patches
>>    are not applied.
>>
>> Paper with short description of deterministic replay implementation:
>> http://www.computer.org/csdl/proceedings/csmr/2012/4666/00/4666a553-abs.html
>>
>> Modifications of qemu include:
>>  * wrappers for clock and time functions to save their return values in the 
>> log
>>  * saving different asynchronous events (e.g. system shutdown) into the log
>>  * synchronization of the bottom halves execution
>>  * synchronization of the threads from thread pool
>>  * recording/replaying user input (mouse and keyboard)
>>  * adding internal events for cpu and io synchronization
>>
>> v8 changes:
>>  * Simplified processing of the shutdown event (as suggested by Paolo 
>> Bonzini)
>>  * Replaced stack of bottom halves in AIO context with QSIMPLEQ (as 
>> suggested by Paolo
>> Bonzini)
>>  * Moved replay_submode out of the series (as suggested by Paolo Bonzini)
>>  * Moved suffix option out of the series
>>  * Converted some of the defines into enums (as suggested by Paolo Bonzini)
>>  * Encapsulated save_tm/read_tm calls into the single function (as suggested 
>> by Paolo Bonzini)
>>  * Moved record/replay options to icount group (as suggested by Paolo 
>> Bonzini)
>>  * Updated mutex protection for the events queue (as suggested by Paolo 
>> Bonzini)
>>  * Added mutex to protect replay log file (as suggested by Paolo Bonzini)
>>  * Minor cleanups
>>
>> v7 changes:
>>  * Removed patches that were applied to upstream.
>>
>> v6 changes:
>>  * Fixed replay stub return value (as suggested by Eric Blake)
>>  * Fixed icount warping.
>>  * Virtual rt clock now uses cpu_get_clock() (as suggested by Paolo Bonzini)
>>  * Replated get_clock_realtime and get_clock calls with qemu clock requests 
>> (as suggested by
>> Paolo Bonzini)
>>  * Modified can_do_io logic to allow requesting icount from cpu_exec 
>> function (as suggested by
>> Paolo Bonzini)
>>  * Removed applied patches.
>>
>> v5 changes:
>>  * Minor changes.
>>  * Used fixed-width integer types for read/write functions (as suggested by 
>> Alex Bennee)
>>  * Moved savevm-related code out of the core.
>>  * Added new traced clock for deterministic virtual clock warping (as 
>> suggested by Paolo
>> Bonzini)
>>  * Fixed exception_index reset for user mode (as suggested by Paolo Bonzini)
>>  * Adopted Paolo's icount patches
>>  * Fixed hardware interrupts replaying
>>
>> v4 changes:
>>  * Updated block drivers to support new bdrv_open interface.
>>  * Moved migration patches into separate series (as suggested by Paolo 
>> Bonzini)
>>  * Fixed a bug in replay_break operation.
>>  * Fixed rtl8139 migration for replay.
>>  * Fixed 'period' parameter processing for record mode.
>>  * Fixed bug in 'reverse-stepi' implementation.
>>  * Fixed replay without making any snapshots (even the starting one).
>>  * Moved core replay patches into the separate series.
>>  * Fixed reverse step and reverse continue support.
>>  * Fixed several bugs in icount subsystem.
>>  * Reusing native qemu icount for replay instructions counting.
>>  * Separated core patches into their own series.
>>
>> v3 changes:
>>  * Fixed bug with replay of the aio write operations.
>>  * Added virtual clock based on replay icount.
>>  * Removed duplicated saving of interrupt_request CPU field.
>>  * Fixed some coding style issues.
>>  * Renamed QMP commands for controlling reverse execution (as suggested by 
>> Eric Blake)
>>  * Replay mode and submode implemented as QAPI enumerations (as suggested by 
>> Eric Blake)
>>  * Added description and example for replay-info command (as suggested by 
>> Eric Blake)
>>  * Added information about the current breakpoint to the output of 
>> replay-info (as suggested
>> by Eric Blake)
>>  * Updated version id for HPET vmstate (as suggested by Paolo Bonzini)
>>  * Removed static fields from parallel vmstate (as suggested by Paolo 
>> Bonzini)
>>  * New vmstate fields for mc146818rtc, pckbd, kvmapic, serial, fdc, rtl8139 
>> moved to
>> subsection (as suggested by Paolo Bonzini)
>>  * Disabled textmode cursor blinking, when virtual machine is stopped (as 
>> suggested by Paolo
>> Bonzini)
>>  * Extracted saving of exception_index to separate patch (as suggested by 
>> Paolo Bonzini)
>>
>> v2 changes:
>>  * Patches are split to be reviewable and bisectable (as suggested by Kirill 
>> Batuzov)
>>  * Added QMP versions of replay commands (as suggested by Eric Blake)
>>  * Removed some optional features of replay to make patches cleaner
>>  * Minor changes and code cleanup were made
>>
>> ---
>>
>> Pavel Dovgalyuk (21):
>>       i386: partial revert of interrupt poll fix
>>       replay: global variables and function stubs
>>       sysemu: system functions for replay
>>       replay: internal functions for replay log
>>       replay: introduce mutex to protect the replay log
>>       replay: introduce icount event
>>       cpu-exec: allow temporary disabling icount
>>       cpu: replay instructions sequence
>>       replay: interrupts and exceptions
>>       replay: asynchronous events infrastructure
>>       replay: recording and replaying clock ticks
>>       replay: recording and replaying different timers
>>       replay: shutdown event
>>       replay: checkpoints
>>       aio: replace stack of bottom halves with queue
>>       replay: bottom halves
>>       replay: replay aio requests
>>       replay: thread pool
>>       replay: initialization and deinitialization
>>       replay: command line options
>>       replay: recording of the user input
>>
>>
>>  Makefile.target                |    1
>>  async.c                        |   49 ++++--
>>  block.c                        |   92 ++++++++++-
>>  block/block-backend.c          |   30 +++-
>>  block/qcow2.c                  |    4
>>  block/raw-posix.c              |    6 -
>>  block/raw-win32.c              |    4
>>  cpu-exec.c                     |   60 ++++++-
>>  cpus.c                         |   45 ++++-
>>  dma-helpers.c                  |   10 +
>>  exec.c                         |    1
>>  hw/block/virtio-blk.c          |   10 +
>>  hw/ide/ahci.c                  |    4
>>  hw/ide/atapi.c                 |   10 +
>>  hw/ide/core.c                  |   18 +-
>>  hw/timer/arm_timer.c           |    2
>>  hw/timer/mc146818rtc.c         |    3
>>  hw/timer/pl031.c               |    3
>>  hw/usb/hcd-uhci.c              |    2
>>  include/block/aio.h            |   22 ++-
>>  include/block/block.h          |   15 ++
>>  include/block/thread-pool.h    |    4
>>  include/exec/exec-all.h        |    2
>>  include/qemu-common.h          |    3
>>  include/qemu/main-loop.h       |    1
>>  include/qemu/queue.h           |    7 +
>>  include/qemu/timer.h           |   16 ++
>>  include/qom/cpu.h              |   10 +
>>  include/sysemu/block-backend.h |   10 +
>>  include/sysemu/cpus.h          |    1
>>  include/ui/input.h             |    2
>>  main-loop.c                    |   10 +
>>  qapi-schema.json               |   18 ++
>>  qemu-io-cmds.c                 |    2
>>  qemu-options.hx                |    8 +
>>  qemu-timer.c                   |   56 +++++--
>>  replay/Makefile.objs           |    5 +
>>  replay/replay-events.c         |  310 ++++++++++++++++++++++++++++++++++++
>>  replay/replay-input.c          |  130 +++++++++++++++
>>  replay/replay-internal.c       |  183 +++++++++++++++++++++
>>  replay/replay-internal.h       |  149 +++++++++++++++++
>>  replay/replay-time.c           |  216 +++++++++++++++++++++++++
>>  replay/replay.c                |  341 
>> ++++++++++++++++++++++++++++++++++++++++
>>  replay/replay.h                |  132 +++++++++++++++
>>  stubs/Makefile.objs            |    1
>>  stubs/replay.c                 |   37 ++++
>>  target-i386/cpu.c              |   10 -
>>  target-i386/seg_helper.c       |    4
>>  tests/test-thread-pool.c       |    7 -
>>  thread-pool.c                  |   49 ++++--
>>  trace-events                   |    2
>>  translate-all.c                |   10 +
>>  ui/input.c                     |   68 ++++++--
>>  util/iov.c                     |    4
>>  vl.c                           |   54 ++++++
>>  55 files changed, 2101 insertions(+), 152 deletions(-)
>>  create mode 100755 replay/Makefile.objs
>>  create mode 100755 replay/replay-events.c
>>  create mode 100755 replay/replay-input.c
>>  create mode 100755 replay/replay-internal.c
>>  create mode 100755 replay/replay-internal.h
>>  create mode 100755 replay/replay-time.c
>>  create mode 100755 replay/replay.c
>>  create mode 100755 replay/replay.h
>>  create mode 100755 stubs/replay.c
>>
>> --
>> Pavel Dovgalyuk
> 
> 
>