Re: [PATCH v2] tests/avocado/tuxrun_baselines.py: improve code coverage for ppc64

[Kautuk Consul]

On 2023-04-20 10:12:37, Thomas Huth wrote:
> On 20/04/2023 09.57, Philippe Mathieu-Daud޸ wrote:
> > Hi Kautuk,
> > 
> > On 19/4/23 11:22, Kautuk Consul wrote:
> > > Commit c0c8687ef0fd990db8db1655a8a6c5a5e35dd4bb disabled the
> > > boot_linux.py test-case due to which the code coverage for ppc
> > > decreased by around 2%. As per the discussion on
> > > https://lore.kernel.org/qemu-devel/87sfdpqcy4.fsf@linaro.org/ it
> > > was mentioned that the baseline test for ppc64 could be modified
> > > to make up this 2% code coverage. This patch attempts to achieve
> > > this 2% code coverage by adding various device command line
> > > arguments (to ./qemu-system-ppc64) in the tuxrun_baselines.py
> > > test-case.
> > > 
> > > The code coverage report with boot_linux.py, without it and finally
> > > with these tuxrun_baselines.py changes is as follows:
> > > 
> > > With boot_linux.py
> > > ------------------
> > > ߦߦ lines......: 13.8% (58006 of 420997 lines)
> > > ߦߦ functions..: 20.7% (7675 of 36993 functions)
> > > ߦߦ branches...: 9.2% (22146 of 240611 branches)
> > > Without boot_linux.py (without this patch changes)
> > > --------------------------------------------------
> > > ߦߦ lines......: 11.9% (50174 of 420997 lines)
> > > ߦߦ functions..: 18.8% (6947 of 36993 functions)
> > > ߦߦ branches...: 7.4% (17580 of 239017 branches)
> > > Without boot_linux.py (with this patch changes)
> > > -----------------------------------------------
> > > ߦߦ lines......: 13.8% (58287 of 420997 lines)
> > > ߦߦ functions..: 20.7% (7640 of 36993 functions)
> > > ߦߦ branches...: 8.4% (20223 of 240611 branches)
> > > 
> > > Signed-off-by: Kautuk Consul <kconsul@linux.vnet.ibm.com>
> > > Reported-by: Alex Benn޸e <alex.bennee@linaro.org>
> > > ---
> > > ߦ tests/avocado/tuxrun_baselines.py | 124 +++++++++++++++++++++++++++++-
> > > ߦ 1 file changed, 122 insertions(+), 2 deletions(-)
> > > 
> > > diff --git a/tests/avocado/tuxrun_baselines.py
> > > b/tests/avocado/tuxrun_baselines.py
> > > index d343376faa..f763ee5d50 100644
> > > --- a/tests/avocado/tuxrun_baselines.py
> > > +++ b/tests/avocado/tuxrun_baselines.py
> > > @@ -11,6 +11,8 @@
> > > ߦ import os
> > > ߦ import time
> > > +import string
> > > +import random
> > > ߦ from avocado import skip, skipIf
> > > ߦ from avocado_qemu import QemuSystemTest
> > > @@ -308,7 +310,7 @@ def test_ppc64(self):
> > > ߦߦߦߦߦߦߦߦߦ """
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=arch:ppc64
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=machine:pseries
> > > -ߦߦߦߦߦߦߦ :avocado: tags=cpu:POWER8
> > > +ߦߦߦߦߦߦߦ :avocado: tags=cpu:POWER10
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=endian:big
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=console:hvc0
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=tuxboot:ppc64
> > > @@ -316,21 +318,139 @@ def test_ppc64(self):
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=extradev:driver=spapr-vscsi
> > > ߦߦߦߦߦߦߦߦߦ :avocado: tags=root:sda
> > > ߦߦߦߦߦߦߦߦߦ """
> > > +ߦߦߦߦߦߦߦ # Generate a random string
> > > +ߦߦߦߦߦߦߦ res = ''.join(random.choices(string.ascii_lowercase +
> > > +ߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦߦ string.digits, k=8))
> > > +
> > > +ߦߦߦߦߦߦߦ # create qcow2 image to be used later.
> > > +ߦߦߦߦߦߦߦ process.run('./qemu-img create -f qcow2 '
> > 
> > Please take qemu-img from $PATH.
> 
> I think it would even be better to provide a function in the base class to
> call qemu-img ... we already have some code in
> tests/avocado/avocado_qemu/__init__.py to find a usable qemu-img binary, I
> think that could be extended?

Thanks everybody for the comments.
I have extended the __init__.py source code to introduce a new function
that finds the qemu-img binary. This function is now getting called from
tuxrun_baselines.py in the v3 patchset I just sent.
I haven't created a function that actually executes qemu-img as there
didn't seem adequate reason to do so.
> 
>  Thomas
>