Re: [Qemu-devel] [PATCH v6 00/25] Fixing record/replay and adding reverse debugging

[dovgaluk]

Can you try applying this patch?
https://www.mail-archive.com/address@hidden/msg563798.html

I also encountered the problems with x86_64 replaying and found the 
misprint in
the code which was fixed later, than sending the series to the mailing 
list.

Pavel Dovgalyuk


Artem Pisarenko писал 2018-10-02 10:02:
> I've added "-monitor stdio" option to command line of Test 1 and
> repeated entering command during execution:
>
>   QEMU 3.0.50 monitor - type 'help' for more information
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 311736195
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 318198367
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 324737211
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 329890795
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 607069789
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 607069789
>   (qemu) info replay
>   Replaying execution 'icount_rr_capture.bin': current step =
> 607069789
>   ...
>
> Some notes on value of step it stucks on:
> - mostly it's same (even across different record-replay pairs);
> - stressing host during replay may cause it to change even for same
> record-replay pair (i.e. different replay executions for same file
> recorded).
>
> This specific case seems to be stable to reproduce.
>
> вт, 2 окт. 2018 г. в 0:22, Artem Pisarenko
> <address@hidden>:
>
> > I've posted bug report with extended tests (incl. case without
> > sleep=off). You may find guest image (kernel) in bug description.
> > https://bugs.launchpad.net/qemu/+bug/1795369 [1]
> >
> > The most annoying thing is that some issues are almost not
> > reproducible. There are definitely race conditions somewhere in qemu
> > code. Running 'stress-ng' utility with CPU and I/O stressors in
> > parallel with qemu execution greatly minimizes amount of attempts
> > when I'm trying to trigger some of issues I encounter.
> >
> > I'll try 'info monitor' command tomorrow, but no guarantees that
> > I'll be able to reproduce issue again.
> >
> > Speaking about '-nographic' and SDL... I've noted that UI greatly
> > minimizes possibility of hanging (but not avoids it completely) when
> > using icount in general, so this effect isn't rr-specific. I've
> > already reported this bug too.
> >
> > пн, 1 окт. 2018 г., 20:14 dovgaluk <address@hidden>:
> >
> > > Artem Pisarenko писал 2018-09-30 14:01:
> > > > Feature still broken :(
> > >
> > > Thanks for testing.
> > >
> > > > Brief description of my tests.
> > > >
> > > > Guest image is Linux, which just powers off after kernel boots
> > > > (instead of proceeding to user-space /init or /sbin/init).
> > > > Base cmdline:
> > > > qemu-system-x86_64 -nodefaults -machine pc,accel=tcg -m 2048
> > > -cpu
> > > > qemu64 -rtc clock=vm,base=2000-01-01T00:00:00 -kernel bzImage
> > > -initrd
> > > > rootfs -append 'nokaslr console=ttyS0 rdinit=/init_poweroff'
> > > > -nographic -serial SERIAL_VALUE -icount
> > > > 1,sleep=off,rr=RR_VALUE,rrfile=icount_rr_capture.bin
> > >
> > > I've never tried it with sleep=off. Can you remove it and try
> > > again?
> > >
> > > We also seen a problem with '-nographic'. When we remove this
> > > option and
> > > QEMU runs with SDL
> > > window, everything is ok. There is some problem with main loop
> > > which may
> > > sleep when there
> > > is no GUI to update, or something like that. We couldn't fix it
> > > yet.
> > >
> > > > Test 1. When SERIAL_VALUE=none
> > > > Running with RR_VALUE=record completes successfully.
> > > > Running with RR_VALUE=replay doesn't completes. qemu process
> > > just
> > > > eating ~100% cpu and memory usage doesn't grow after some
> > > moment. I
> > > > don't see what happens because of problem no.2 (see below).
> > >
> > > Try 'info replay' monitor command. Does instruction counter
> > > increases?
> > >
> > > > Test 2. When SERIAL_VALUE=stdio
> > > > Running with RR_VALUE=record completes successfully.
> > > >
> > > > Running with RR_VALUE=replay caues exit with error:
> > > >
> > > > "qemu-system-x86_64: Missing character write event in the replay
> > > log"
> > > > These problems are same with qemu 2.12 (both vanilla and with
> > > previous
> > > > versions of these patches applied). Furthemore, I consider whole
> > > > icount mode broken and determinism isn't achievable.
> > > > The irony is that I actually don't need record/replay feature.
> > > I've
> > > > tried to use it only as instrument to debug failing determinism
> > > in
> > > > qemu code. But since replay/record feature itself relies on
> > > > determinism, which is broken, it's no wonder why it fails also
> > > (I just
> > > > hoped to bypass it).
> > > >
> > > > Contact me if you need more details. I just tired a lot trying
> > > to get
> > > > all these things working... Hope is leaving me...
> > >
> > > Can you share the kernel in case the icount still broken?
> > >
> > > Pavel Dovgalyuk
> > --
> >
> > С уважением,
> > Артем Писаренко
>  --
>
> С уважением,
>   Артем Писаренко
>
> Links:
> ------
> [1] https://bugs.launchpad.net/qemu/+bug/1795369