Re: [PATCH 1/6] tests/boot_linux_console: add microvm acceptance test

[Wainer dos Santos Moschetta]

On 1/30/20 9:51 PM, Philippe Mathieu-Daudé wrote:
> On 1/30/20 6:41 PM, Wainer dos Santos Moschetta wrote:
> > On 1/27/20 2:36 PM, Liam Merwick wrote:
> > > Refactor test_x86_64_pc() to test_x86_64_machine() so that separate
> > > functions which specify the Avocado tag of ':avocado: tags=machine:'
> > > as being either 'pc' or 'microvm' can be used to test booting a
> > > compressed kernel using either machine class.
> > >
> > > Signed-off-by: Liam Merwick <address@hidden>
> > > ---
> > >   tests/acceptance/boot_linux_console.py | 15 +++++++++++++--
> > >   1 file changed, 13 insertions(+), 2 deletions(-)
> > >
> > > diff --git a/tests/acceptance/boot_linux_console.py 
> > > b/tests/acceptance/boot_linux_console.py
> > > index e40b84651b0b..aa5b07b1c609 100644
> > > --- a/tests/acceptance/boot_linux_console.py
> > > +++ b/tests/acceptance/boot_linux_console.py
> > > @@ -51,10 +51,9 @@ class BootLinuxConsole(Test):
> > >           os.chdir(cwd)
> > >           return self.workdir + path
> > > -    def test_x86_64_pc(self):
> > > +    def do_test_x86_64_machine(self):
> > >           """
> > >           :avocado: tags=arch:x86_64
> > > -        :avocado: tags=machine:pc
> > >           """
> > >           kernel_url = 
> > > ('https://archives.fedoraproject.org/pub/archive/fedora'
> > > '/linux/releases/29/Everything/x86_64/os/images/pxeboot'
> > > @@ -70,6 +69,18 @@ class BootLinuxConsole(Test):
> > >           console_pattern = 'Kernel command line: %s' % 
> > > kernel_command_line
> > >           self.wait_for_console_pattern(console_pattern)
> > > +    def test_x86_64_pc(self):
> > > +        """
> > > +        :avocado: tags=machine:pc
> > > +        """
> >
> > The test method won't inherit the 'arch' tag from 
> > `do_test_x86_64_machine()`, so you need to explicitly 'arch' tag each 
> > test you created in this series. If you don't do so, Avocado won't 
> > filter out those x86_64 tests in case QEMU is built with non-x86_64 
> > targets.
> >
> > Follows an example, I built QEMU with '--target-list=arm-softmmu'. I 
> > got:
> >
> > ```
> >
> > (02/18) 
> > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_pc: 
> > CANCEL: No QEMU binary defined or found in the source tree (0.00 s)
> >   (03/18) 
> > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_x86_64_microvm: 
> > CANCEL: No QEMU binary defined or found in the source tree (0.00 s)
> >   (04/18) 
> > tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_virt: 
> > PASS (1.25 s)
> >
> > ```
> >
> > OK, avocado_qemu was smart enough to skip the tests, but ideally it 
> > should not even consider running them in the first place.
>
> This should be solved by this patch:
> https://lists.gnu.org/archive/html/qemu-devel/2020-01/msg07311.html
> "tests/acceptance: Use 'machine' tag to check if available in QEMU binary"


That patch is useful**  but does not solve the problem that I point out 
here. If you have a test case which is arch-specific and the arch-QEMU 
was not built then it should not be executed. Tagging the test with 
'arch' ensures that.

** BTW, I'm going to review it soon

- Wainer


> > Thanks!
> >
> > - Wainer
> >
> > > +        self.do_test_x86_64_machine()
> > > +
> > > +    def test_x86_64_microvm(self):
> > > +        """
> > > +        :avocado: tags=machine:microvm
> > > +        """
> > > +        self.do_test_x86_64_machine()
> > > +
> > >       def test_mips_malta(self):
> > >           """
> > >           :avocado: tags=arch:mips