[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [qemu-s390x] [PATCH v2 02/20] Acceptance tests: show avocado test ex

From: Cleber Rosa
Subject: Re: [qemu-s390x] [PATCH v2 02/20] Acceptance tests: show avocado test execution by default
Date: Wed, 6 Feb 2019 12:02:55 -0500
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0

On 2/6/19 9:36 AM, Cornelia Huck wrote:
> On Fri,  1 Feb 2019 19:55:52 -0500
> Cleber Rosa <address@hidden> wrote:
>> The current version of the "check-acceptance" target will only show
>> one line for execution of all tests.  That's probably OK if the tests
>> to be run are quick enough and they're always the same.
>> But, there's already one test alone that takes on average ~5 seconds
>> to run, we intend to adapt the list of tests to match the user's build
>> environment (among other choices).
> Btw: What are our expectations regarding execution time for tests?
> Especially if we continue adding tests, and architecture-specific tests
> are bound to be slower if run on a foreign architecture via tcg.
> Would a make check-acceptance-quick command make sense? ("I only want
> to verify quickly that I didn't break too much, so run the quicker
> tests only, probably only for my host architecture")

Yes, it definitely makes sense.  Now, let me know if the following also
makes sense to you:

1) Because these tests focus on functional testing, the default
target/shortcut ("make check-acceptance") should run the complete set of
test cases (including the slow ones).

2) Requirements vary greatly from user to user, to while adding a
"check-acceptance-quick" is fine, you just mentioned one extra test
execution variation ("for my host architecture").  For those, the idea
is that:

 a) "make check-acceptance[-quick]" will adapt to the build environment
(if you only built s390x targets, that's all it's going to use)

 b) "avocado" command line interface *should* be easy enough to fulfill
other requirements, and not necessarily require a "make" target.  For
instance, if you're only interested in your host arch and one specific
machine type, a command line such as the following should do the trick:

  $ make check-venv
  $ ./tests/venv/bin/avocado run -t arch:`uname -m` -t
machine:WHAT_I_CARE_ABOUT tests/acceptance/

How does that sound?

- Cleber.

>> Because of that, let's present the default Avocado UI by default.
>> Users can always choose a different output by setting the AVOCADO_SHOW
>> variable.
>> Reviewed-by: Caio Carrara <address@hidden>
>> Reviewed-by: Philippe Mathieu-Daudé <address@hidden>
>> Reviewed-by: Alex Bennée <address@hidden>
>> Signed-off-by: Cleber Rosa <address@hidden>
>> ---
>>  .travis.yml            | 2 +-
>>  tests/Makefile.include | 2 +-
>>  2 files changed, 2 insertions(+), 2 deletions(-)

reply via email to

[Prev in Thread] Current Thread [Next in Thread]