Re: [Qemu-devel] [PATCH v2 4/6] tests/acceptance: Add a BootLinuxConsoleMips test

[Philippe Mathieu-Daudé]

On 06/28/2018 01:23 PM, Alex Bennée wrote:
> 
> Philippe Mathieu-Daudé <address@hidden> writes:
> 
>> Similar to the BootLinuxConsoleX86_64 test:
>> boot a Linux kernel on a Malta board and verify the serial is working.
>>
>> This test can be run using:
>>
>>     $ avocado run -t endian:big tests/acceptance
>>
>> Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
>> ---
>>  tests/acceptance/boot_linux_console.py | 38 ++++++++++++++++++++++++++
>>  1 file changed, 38 insertions(+)
>>
>> diff --git a/tests/acceptance/boot_linux_console.py 
>> b/tests/acceptance/boot_linux_console.py
>> index 17dc8d58c1..72cf5e943c 100644
>> --- a/tests/acceptance/boot_linux_console.py
>> +++ b/tests/acceptance/boot_linux_console.py
>> @@ -46,3 +46,41 @@ class BootLinuxConsoleX86_64(Test):
>>                  break
>>              if 'Kernel panic - not syncing' in msg:
>>                  self.fail("Kernel panic reached")
>> +
>> +
>> +class BootLinuxConsoleMips(Test):
>> +    """
>> +    Boots a mips Linux kernel and checks that the console is operational
>> +    and the kernel command line is properly passed from QEMU to the kernel
>> +
>> +    :avocado: enable
>> +    :avocado: tags=endian:big
>> +    :avocado: tags=arch:mips
>> +    :avocado: tags=board:malta
>> +    """
>> +
>> +    arch = "mips"
>> +    timeout = 60
>> +
>> +    def test(self):
>> +        kernel_url = ('http://people.debian.org/~aurel32/qemu/mips/'
>> +                      'vmlinux-3.2.0-4-4kc-malta')
>> +        kernel_hash = '592e384a4edc16dade52a6cd5c785c637bcbc9ad'
>> +        kernel_path = self.fetch_asset(kernel_url,
>> asset_hash=kernel_hash)
> 
> I'm uncomfortable using "random" binaries of websites as the source of
> our test kernels. I can see the justification for distro kernels as they
> at least have the infrastructure to rebuild from source if you really
> want to, but even then the distros don't cover a lot of the
> architectures.

And now I notice I made an mistake here :) I guess remember the Avocado
team started using the SHA-1 hash as default and I suggested them to be
able to use other hashes for this particular case, since Aurelien
provided the MD5 hashes signed by his GPG key, which is signed/trusted
by Peter and used to merge mips32 pulls.

That would verify the QEMU community circle of trust right?

I don't think Avocado should parse the FTP/HTTP signed indexes, but a
manual verification when merging this series should suffice.

> 
> I had experimented with using docker based builds for building test
> fixtures (see tests/docker/dockerbuilds):
> 
>    https://github.com/stsquad/qemu/tree/docker/linux-user-and-ltp-builds-v2
> 
> As these tests are fairly simple boot tests that just need kernels maybe
> we could look at tooling up the generation of these images in a
> repeatable way - similar to the way vmtest builds VMs.

Yes, I have another acceptance branch where I cross-build an old mipssim
kernel to test the board, using the following:
http://lists.nongnu.org/archive/html/qemu-devel/2018-04/msg04071.html

But preparing a Docker cross image, fetching the Linux kernel source,
building it, takes a lot of time/storage I'd rather avoid; at least with
Aurelien kernels, since they are known to work since years.

>> +
>> +        self.vm.set_machine('malta')
>> +        self.vm.set_console()
>> +        kernel_command_line = 'console=ttyS0 printk.time=0'
>> +        self.vm.add_args('-m', "64",
>> +                         '-kernel', kernel_path,
>> +                         '-append', kernel_command_line)
>> +        self.vm.launch()
>> +        console = self.vm.console_socket.makefile()
>> +        console_logger = logging.getLogger('console')
>> +        while True:
>> +            msg = console.readline()
>> +            console_logger.debug(msg.strip())
>> +            if 'Kernel command line: %s' % kernel_command_line in msg:
>> +                break
>> +            if 'Kernel panic - not syncing' in msg:
>> +                self.fail("Kernel panic reached")
> 
> Of course for bonus points a simple rootfs with hackbench or some such
> in it would be nice. But I appreciate this makes the building job a lot
> more complex than just a kernel.

My idea is to use the rootfs for larger tests, and tag the "Kernel
panic" tests as "quick", so we can have a "make acceptance-speed" or
similar.

We can already test than many devices were initialized correctly quickly
looking at this console output.

Regards,

Phil.