qemu-commits
[Top][All Lists]
Advanced
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Qemu-commits] [qemu/qemu] 59692a: tests/acceptance: show avocado test e

From: Peter Maydell
Subject: [Qemu-commits] [qemu/qemu] 59692a: tests/acceptance: show avocado test execution by d...
Date: Fri, 03 May 2019 08:02:09 -0700 
  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 59692a1238f7f1e0afa85f67ae15a664f786ece0
      
https://github.com/qemu/qemu/commit/59692a1238f7f1e0afa85f67ae15a664f786ece0
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/Makefile.include

  Log Message:
  -----------
  tests/acceptance: show avocado test execution by default

The current version of the "check-acceptance" target will only show
one line for execution of all tests.  That's probably OK if the tests
to be run are quick enough and they're always the same.

But, there's already one test alone that takes on average ~5 seconds
to run, we intend to adapt the list of tests to match the user's build
environment (among other choices).

Because of that, let's present the default Avocado UI by default.
Users can always choose a different output by setting the AVOCADO_SHOW
variable.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 45c01bd926ec072a9ec54b2fdd780917dd7786b5
      
https://github.com/qemu/qemu/commit/45c01bd926ec072a9ec54b2fdd780917dd7786b5
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/acceptance/avocado_qemu/__init__.py

  Log Message:
  -----------
  tests/acceptance: improve docstring on pick_default_qemu_bin()

Making it clear what is returned by this utility function.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 805fac5272825be4a241d028a4ac02d47355b891
      
https://github.com/qemu/qemu/commit/805fac5272825be4a241d028a4ac02d47355b891
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M docs/devel/testing.rst

  Log Message:
  -----------
  tests/acceptance: fix doc reference to avocado_qemu directory

The "this directory" reference is misleading and confusing, it's a
leftover from when this text was proposed in a README file inside
the "tests/acceptance/avocado_qemu" directory.

When that text was moved to the top level docs directory, the
reference was not updated.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 2c44d68f2b004e41136996fef0e4122e6507ba90
      
https://github.com/qemu/qemu/commit/2c44d68f2b004e41136996fef0e4122e6507ba90
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M docs/devel/testing.rst
    M tests/acceptance/avocado_qemu/__init__.py

  Log Message:
  -----------
  tests/acceptance: introduce arch parameter and attribute

It's useful to define the architecture that should be used in
situations such as:
 * the intended target of the QEMU binary to be used on tests
 * the architecture of code to be run within the QEMU binary, such
   as a kernel image or a full blown guest OS image

This commit introduces both a test parameter and a test instance
attribute, that will contain such a value.

Now, when the "arch" test parameter is given, it will influence the
selection of the default QEMU binary, if one is not given explicitly
by means of the "qemu_img" parameter.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: b194713f94b286fd1f2bccace45cb5b25316e232
      
https://github.com/qemu/qemu/commit/b194713f94b286fd1f2bccace45cb5b25316e232
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/Makefile.include
    M tests/acceptance/boot_linux_console.py
    M tests/acceptance/linux_initrd.py
    M tests/acceptance/virtio_version.py

  Log Message:
  -----------
  tests/acceptance: use "arch:" tag to filter target specific tests

Currently, some tests contains target architecture information, in the
form of a "x86_64" tag.  But that tag is not respected in the default
execution, that is, "make check-acceptance" doesn't do anything with
it.

That said, even the target architecture handling currently present in
the "avocado_qemu.Test" class is pretty limited.  For instance, by
default, it chooses a target based on the host architecture.

Because the original implementation of the tags feature in Avocado did
not include any time of namespace or "key:val" mechanism, no tag has
relation to another tag.  The new implementation of the tags feature
from version 67.0 onwards, allows "key:val" tags, and because of that,
a test can be classified with a tag in a given key.  For instance, the
new proposed version of the "boot_linux_console.py" test, which
downloads and attempts to run a x86_64 kernel, is now tagged as:

  :avocado: tags=arch:x86_64

This means that it can be filtered (out) when no x86_64 target is
available.  At the same time, tests that don't have a "arch:" tag,
will not be filtered out.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: b910545fbf3f55f22545c202c1aee62598edb838
      
https://github.com/qemu/qemu/commit/b910545fbf3f55f22545c202c1aee62598edb838
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M docs/devel/testing.rst
    M tests/acceptance/avocado_qemu/__init__.py

  Log Message:
  -----------
  tests/acceptance: look for target architecture in test tags first

A test can, optionally, be tagged for one or many architectures.  If a
test has been tagged for a single architecture, there's a high chance
that the test won't run on other architectures.  This changes the
default order of choosing a default target architecture to use based
on the 'arch' tag value first.

The precedence order is for choosing a QEMU binary to use for a test
is now:

 * qemu_bin parameter
 * arch parameter
 * arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64
   is used)

This means that if one runs:

 $ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py

No arch parameter or tag will influence the selection of the QEMU
target binary.  If one runs:

 $ avocado run -p arch=ppc64 test.py

The target binary selection mechanism will attempt to find a binary
such as "ppc64-softmmu/qemu-system-ppc64".  And finally, if one runs
a test that is tagged (in its docstring) with "arch:aarch64":

 $ avocado run aarch64.py

The target binary selection mechanism will attempt to find a binary
such as "aarch64-softmmu/qemu-system-aarch64".

At this time, no provision is made to cancel the execution of tests if
the arch parameter given (manually) does not match the test "arch"
tag, but it may be a useful default behavior to be added in the
future.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 78664ed84949fef4cc0a8be45dd1968e5713e18c
      
https://github.com/qemu/qemu/commit/78664ed84949fef4cc0a8be45dd1968e5713e18c
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: rename the x86_64 after the arch and machine

Given that the test is specific to x86_64 and pc, and new tests are
going to be added to the same class, let's rename it accordingly.
Also, let's make the class documentation not architecture specific.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 7d7985b1613345541df7c7fc032e470c71cba942
      
https://github.com/qemu/qemu/commit/7d7985b1613345541df7c7fc032e470c71cba942
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: update the x86_64 kernel

Update to the stock Fedora 29 kernel, from the Fedora 28.  New tests
will be added using the 29 kernel, so for consistency, let's also
update it here.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Tested-by: Philippe Mathieu-Daudé <address@hidden>
Acked-by: Cornelia Huck <address@hidden>
CC: Daniel P. Berrangé <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: b50fcd39ba077465852870c17687df17227759a5
      
https://github.com/qemu/qemu/commit/b50fcd39ba077465852870c17687df17227759a5
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add common kernel command line options

The 'printk.time=0' option makes it easier to parse the console
output.  Let's set it as a default, and reusable, kernel command line
options for this and future similar tests.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 61f74506623186f076233dc5d5a8d92f4f8235dd
      
https://github.com/qemu/qemu/commit/61f74506623186f076233dc5d5a8d92f4f8235dd
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: increase timeout

When running on very low powered environments, some tests may time out
causing false negatives.  As a conservative change, and for
considering that human time (investigating false negatives) is worth
more than some extra machine cycles (and time), let's increase the
overall timeout.

CC: Alex Bennée <address@hidden>
Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 0d1d74e5e572034abb90c5b187ff3b2edd778960
      
https://github.com/qemu/qemu/commit/0d1d74e5e572034abb90c5b187ff3b2edd778960
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: refactor the console watcher into utility method

This introduces a utility method that monitors the console device and
looks for either a message that signals the test success or failure.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 123990adbf93043a5ef061be0667a23afe5d8abb
      
https://github.com/qemu/qemu/commit/123990adbf93043a5ef061be0667a23afe5d8abb
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M python/qemu/__init__.py

  Log Message:
  -----------
  scripts/qemu.py: support adding a console with the default serial device

The set_console() utility function either adds a device based on the
explicitly given device type, or adds a known good type of device
based on the machine type.

But, for a number of machine types, it may be impossible or
inconvenient to add the devices by means of "-device" command line
options, and then it may better to just use the "-serial" option and
let QEMU itself, based on the machine type, set the device
accordingly.

To achieve that, the behavior of set_console() now flags the intention
to add a console device on launch(), and if no explicit device type is
given the "-serial" option is going to be added to the QEMU command
line, instead of raising exceptions.

Based on testing with different machine types, the CONSOLE_DEV_TYPES
is not necessary anymore, so it's being removed, as is the logic to
use it.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: f87920474d29e48c7548e6e826c3d50a259f2599
      
https://github.com/qemu/qemu/commit/f87920474d29e48c7548e6e826c3d50a259f2599
  Author: Philippe Mathieu-Daudé <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add a test for mips + malta

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta
board and verify the serial is working.  Also, it relies on the serial
device set by the machine itself.

If mips is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:mips" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips tests/acceptance
    $ avocado run -t machine:malta tests/acceptance
    $ avocado run -t endian:big tests/acceptance

Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Aleksandar Markovic <address@hidden>
Tested-by: Philippe Mathieu-Daudé <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 02c2852bcd8fadea3d81c380bdaeaee1ae91247f
      
https://github.com/qemu/qemu/commit/02c2852bcd8fadea3d81c380bdaeaee1ae91247f
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add a test for mips64el + malta

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta
board and verify the serial is working.

If mips64el is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:mips64el"
tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:mips64el tests/acceptance
    $ avocado run -t machine:malta tests/acceptance

Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Aleksandar Markovic <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: d4e12161677bbad86079d15236f1d449cc860978
      
https://github.com/qemu/qemu/commit/d4e12161677bbad86079d15236f1d449cc860978
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add a test for aarch64 + virt

Just like the previous tests, boots a Linux kernel on a aarch64 target
using the virt machine.

One special option added is the CPU type, given that the kernel
selected fails to boot on the virt machine's default CPU (cortex-a15).

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 1a30892ed5f253f6bf7adbe56ef5931bd69cd4b6
      
https://github.com/qemu/qemu/commit/1a30892ed5f253f6bf7adbe56ef5931bd69cd4b6
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add a test for arm + virt

Just like the previous tests, boots a Linux kernel on an arm target
using the virt machine.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: 7918249416dddc5ab2d4bcb1e6c2742f3be0edac
      
https://github.com/qemu/qemu/commit/7918249416dddc5ab2d4bcb1e6c2742f3be0edac
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add a test for s390x + s390-ccw-virtio

Just like the previous tests, boots a Linux kernel on a s390x target
using the s390-ccw-virtio machine.

Because it's not possible to have multiple VT220 consoles,
'-nodefaults' is used, so that the one set with set_console() works
correctly.

Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Cornelia Huck <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
[ehabkost: Updated kernel URL to point to fedoraproject.org]
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: b36b59371fbb39ae39645b16db85f9a49573309d
      
https://github.com/qemu/qemu/commit/b36b59371fbb39ae39645b16db85f9a49573309d
  Author: Cleber Rosa <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M .travis.yml
    M tests/acceptance/boot_linux_console.py

  Log Message:
  -----------
  tests/boot_linux_console: add a test for alpha + clipper

Similar to the x86_64 + pc test, it boots a Linux kernel on a Malta
board and verify the serial is working.  One extra command added to
the QEMU command line is '-vga std', because the kernel used is
known to crash without it.

If alpha is a target being built, "make check-acceptance" will
automatically include this test by the use of the "arch:alpha" tags.

Alternatively, this test can be run using:

    $ avocado run -t arch:alpha tests/acceptance
    $ avocado run -t machine:clipper tests/acceptance

Signed-off-by: Philippe Mathieu-Daudé <address@hidden>
Signed-off-by: Cleber Rosa <address@hidden>
Reviewed-by: Caio Carrara <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: faf441429adfe5767be52c5dcdb8bc03161d064f
      
https://github.com/qemu/qemu/commit/faf441429adfe5767be52c5dcdb8bc03161d064f
  Author: Daniel P. Berrangé <address@hidden>
  Date:   2019-05-02 (Thu, 02 May 2019)

  Changed paths:
    M configure

  Log Message:
  -----------
  configure: automatically pick python3 is available

Unless overridden via an env var or configure arg, QEMU will only look
for the 'python' binary in $PATH. This is unhelpful on distros which
are only shipping Python 3.x (eg Fedora) in their default install as,
if they comply with PEP 394, the bare 'python' binary won't exist.

This changes configure so that by default it will search for all three
common python binaries, preferring to find Python 3.x versions.

Signed-off-by: Daniel P. Berrangé <address@hidden>
Message-Id: <address@hidden>
Signed-off-by: Eduardo Habkost <address@hidden>


  Commit: a6ae23831b05a11880b40f7d58e332c45a6b04f7
      
https://github.com/qemu/qemu/commit/a6ae23831b05a11880b40f7d58e332c45a6b04f7
  Author: Peter Maydell <address@hidden>
  Date:   2019-05-03 (Fri, 03 May 2019)

  Changed paths:
    M .travis.yml
    M configure
    M docs/devel/testing.rst
    M python/qemu/__init__.py
    M tests/Makefile.include
    M tests/acceptance/avocado_qemu/__init__.py
    M tests/acceptance/boot_linux_console.py
    M tests/acceptance/linux_initrd.py
    M tests/acceptance/virtio_version.py

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ehabkost/tags/python-next-pull-request' 
into staging

Python queue, 2019-05-02

* configure: automatically pick python3 is available
  (Daniel P. Berrangé)

* tests/acceptance (Cleber Rosa, Philippe Mathieu-Daudé):
  * Multi-architecture test support
  * Multiple arch-specific boot_linux_console test cases
  * Increase verbosity of avocado by default
  * docstring improvements

# gpg: Signature made Fri 03 May 2019 01:40:06 BST
# gpg:                using RSA key 2807936F984DC5A6
# gpg: Good signature from "Eduardo Habkost <address@hidden>" [full]
# Primary key fingerprint: 5A32 2FD5 ABC4 D3DB ACCF  D1AA 2807 936F 984D C5A6

* remotes/ehabkost/tags/python-next-pull-request:
  configure: automatically pick python3 is available
  tests/boot_linux_console: add a test for alpha + clipper
  tests/boot_linux_console: add a test for s390x + s390-ccw-virtio
  tests/boot_linux_console: add a test for arm + virt
  tests/boot_linux_console: add a test for aarch64 + virt
  tests/boot_linux_console: add a test for mips64el + malta
  tests/boot_linux_console: add a test for mips + malta
  scripts/qemu.py: support adding a console with the default serial device
  tests/boot_linux_console: refactor the console watcher into utility method
  tests/boot_linux_console: increase timeout
  tests/boot_linux_console: add common kernel command line options
  tests/boot_linux_console: update the x86_64 kernel
  tests/boot_linux_console: rename the x86_64 after the arch and machine
  tests/acceptance: look for target architecture in test tags first
  tests/acceptance: use "arch:" tag to filter target specific tests
  tests/acceptance: introduce arch parameter and attribute
  tests/acceptance: fix doc reference to avocado_qemu directory
  tests/acceptance: improve docstring on pick_default_qemu_bin()
  tests/acceptance: show avocado test execution by default

Signed-off-by: Peter Maydell <address@hidden>

# Conflicts:
#       configure


Compare: https://github.com/qemu/qemu/compare/5b396a8c36f4...a6ae23831b05
reply via email to
[Prev in Thread] Current Thread [Next in Thread]