Re: [PATCH] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

[Kautuk Consul]

On 2023-04-18 09:07:53, Thomas Huth wrote:
> On 18/04/2023 07.53, Kautuk Consul wrote:
> > Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the
> > boot_linux.py test-case due to which the code coverage for ppc
> > decreased by around 2%. As per the discussion on
> > https://lore.kernel.org/qemu-devel/87sfdpqcy4.fsf@linaro.org/ it
> > was mentioned that the baseline test for ppc64 could be modified
> > to make up this 2% code coverage. This patch attempts to achieve
> > this 2% code coverage by adding various device command line
> > arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py
> > test-case.
> > 
> > The code coverage report with boot_linux.py, without it and finally
> > with these tuxrun_baselines.py changes is as follows:
> > 
> > With boot_linux.py
> > ------------------
> >    lines......: 13.8% (58006 of 420997 lines)
> >    functions..: 20.7% (7675 of 36993 functions)
> >    branches...: 9.2% (22146 of 240611 branches)
> > Without boot_linux.py (without this patch changes)
> > --------------------------------------------------
> >    lines......: 11.9% (50174 of 420997 lines)
> >    functions..: 18.8% (6947 of 36993 functions)
> >    branches...: 7.4% (17580 of 239017 branches)
> > Without boot_linux.py (with this patch changes)
> > -----------------------------------------------
> >    lines......: 13.8% (58287 of 420997 lines)
> >    functions..: 20.7% (7640 of 36993 functions)
> >    branches...: 8.4% (20223 of 240611 branches)
> > 
> > Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
> > Reported-by: Alex Bennée <alex.bennee@linaro.org>
> > ---
> >   tests/avocado/tuxrun_baselines.py | 98 ++++++++++++++++++++++++++++++-
> >   1 file changed, 96 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tests/avocado/tuxrun_baselines.py 
> > b/tests/avocado/tuxrun_baselines.py
> > index d343376faa..cb17602c94 100644
> > --- a/tests/avocado/tuxrun_baselines.py
> > +++ b/tests/avocado/tuxrun_baselines.py
> > @@ -308,7 +308,7 @@ def test_ppc64(self):
> >           """
> >           :avocado: tags=arch:ppc64
> >           :avocado: tags=machine:pseries
> > -        :avocado: tags=cpu:POWER8
> > +        :avocado: tags=cpu:POWER10
> >           :avocado: tags=endian:big
> >           :avocado: tags=console:hvc0
> >           :avocado: tags=tuxboot:ppc64
> > @@ -316,19 +316,113 @@ def test_ppc64(self):
> >           :avocado: tags=extradev:driver=spapr-vscsi
> >           :avocado: tags=root:sda
> >           """
> > +        self.vm.add_args('-netdev', 
> > 'user,id=vnet,hostfwd=:127.0.0.1:0-:22',
> > +                         '-device', 'virtio-net,netdev=vnet')
> > +        self.vm.add_args('-netdev', '{"type":"user","id":"hostnet0"}',
> > +                         '-device', '{"driver":"virtio-net-pci","netdev":'
> > +                         
> > '"hostnet0","id":"net0","mac":"52:54:00:4c:e3:86",'
> 
> The patch looks already fine to me as it is, but in case you'd want to
> increase test coverage even a little bit more, you could add some lines that
> reads out the MAC address again from the /sys fs in the guest to check that
> it matches the one that has been specified here. See
> tests/avocado/machine_s390_ccw_virtio.py as an example, we're doing it for
> the s390x machine there already.
The goal was to achieve the same level of code coverage as what we had
before skipping the boot_linux.py. So I'll let this be for now. Of
course we can improve the code coverage further if required later.
> 
> > +        process.run('./qemu-img create -f qcow2'
> > +                    '/tmp/tuxrun_baselines_ppc64le.qcow2 1G')
> 
> Please clean up temporary files after the test has finished.
> Also it might be necessary to randomize the file name to make it possible
> that multiple instances of the test can be run in parallel.
Will make these changes and post a v2.

Thanks!
> 
>  Thomas
>