Re: [PATCH v2 03/11] tests/acceptance: add base class record/replay kernel tests

[Pavel Dovgalyuk]

On 28.05.2020 11:28, Alex Bennée wrote:
> Pavel Dovgalyuk <dovgaluk@ispras.ru> writes:
>
> > On 27.05.2020 18:20, Alex Bennée wrote:
> > > Pavel Dovgalyuk <Pavel.Dovgaluk@gmail.com> writes:
> > >
> > > > This patch adds a base for testing kernel boot recording and replaying.
> > > > Each test has the phase of recording and phase of replaying.
> > > > Virtual machines just boot the kernel and do not interact with
> > > > the network.
> > > > Structure and image links for the tests are borrowed from boot_linux_console.py
> > > > Testing controls the message pattern at the end of the kernel
> > > > boot for both record and replay modes. In replay mode QEMU is also
> > > > intended to finish the execution automatically.
> > > >
> > > > Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> > > diff --git a/MAINTAINERS b/MAINTAINERS
> > > index 47ef3139e6..e9a9ce4f66 100644
> > > --- a/MAINTAINERS
> > > +++ b/MAINTAINERS
> > > @@ -2497,6 +2497,7 @@ F: net/filter-replay.c
> > >    F: include/sysemu/replay.h
> > >    F: docs/replay.txt
> > >    F: stubs/replay.c
> > > +F: tests/acceptance/replay_kernel.py
> > >      IOVA Tree
> > >    M: Peter Xu <peterx@redhat.com>
> > > diff --git a/tests/acceptance/replay_kernel.py 
> > > b/tests/acceptance/replay_kernel.py
> > > new file mode 100644
> > > index 0000000000..b8b277ad2f
> > > --- /dev/null
> > > +++ b/tests/acceptance/replay_kernel.py
> > > @@ -0,0 +1,57 @@
> > > +# Record/replay test that boots a Linux kernel
> > > +#
> > > +# Copyright (c) 2020 ISP RAS
> > > +#
> > > +# Author:
> > > +#  Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
> > > +#
> > > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > > +# later.  See the COPYING file in the top-level directory.
> > > +
> > > +import os
> > > +import gzip
> > >
> > > Do we actually use gzip in this test?
> > Removed that, thanks.
> >
> > > +
> > > +from avocado_qemu import wait_for_console_pattern
> > > +from avocado.utils import process
> > > +from avocado.utils import archive
> > > +from boot_linux_console import LinuxKernelUtils
> > > +
> > > +class ReplayKernel(LinuxKernelUtils):
> > > +    """
> > > +    Boots a Linux kernel in record mode and checks that the console
> > > +    is operational and the kernel command line is properly passed
> > > +    from QEMU to the kernel.
> > > +    Then replays the same scenario and verifies, that QEMU correctly
> > > +    terminates.
> > >
> > > Shouldn't we be doing more to verify the replay behaved the same as the
> > > recorded session? What happens if things go wrong? Does QEMU barf out or
> > > just deviate from the previous run?
> > We hardly can compare vCPU states during record and replay.
> >
> > But in the most cases it is not needed. When control flow goes in the
> > wrong direction, it affects the interrupts and exceptions.
> >
> > And interrupts and exceptions are the synchronization points in the
> > replay log. Therefore when the executions differ, QEMU replay just
> > hangs.
> Maybe we should fix that and exit with a more definitive error? Hangs
> are just plain ugly to debug because your first step has to be to start
> poking around with a debugger.
Good point, I'll thinks about it.