Re: [Qemu-devel] [PATCH 6/7] Acceptance Tests: add variants definition for architectures

[Cleber Rosa]

On 10/5/18 1:30 PM, Philippe Mathieu-Daudé wrote:
> On 05/10/2018 19:07, Cleber Rosa wrote:
>> On 10/5/18 12:32 PM, Eric Blake wrote:
>>> On 10/5/18 11:24 AM, Philippe Mathieu-Daudé wrote:
>>>> Hi Cleber,
>>>>
>>>> On 04/10/2018 17:14, Cleber Rosa wrote:
>>>>> One of the Avocado features relevant to virtualization testing is the
>>>>> ability to reuse tests in different scenarios, known as variants.
>>>>> This adds a JSON based variants file, that can be used to run most
>>>>> tests in a number of different architectures.  It can be run with:
>>>>>
>>>>>     $ avocado run \
>>>>>       --json-variants-load=tests/acceptance/variants/arch.json \
>>>>>       --filter-by-tags='-x86_64' -- tests/acceptance/
>>>
>>>>> +++ b/tests/acceptance/variants/arch.json
>>>>> @@ -0,0 +1 @@
>>>>> +[{"paths":["/run/*"],"variant":[["/run/aarch64",[["/run/aarch64",
>>>>> "arch", "aarch64"]]]],"variant_id":
>>>>> "aarch64"},{"paths":["/run/*"],"variant":[["/run/ppc",[["/run/ppc",
>>>>> "arch", "ppc"]]]],"variant_id":
>>>>> "ppc"},{"paths":["/run/*"],"variant":[["/run/ppc64",[["/run/ppc64",
>>>>> "arch", "ppc64"]]]],"variant_id":
>>>>> "ppc64"},{"paths":["/run/*"],"variant":[["/run/s390x",[["/run/s390x",
>>>>> "arch", "s390x"]]]],"variant_id":
>>>>> "s390x"},{"paths":["/run/*"],"variant":[["/run/x86_64",[["/run/x86_64",
>>>>> "arch", "x86_64"]]]],"variant_id": "x86_64"}]
>>>>>
>>>>
>>>> Is this generated? (thinking about the other archs supported).
>>>>
>>
>> It's generated and kept on every job result (jobdata/variants.json).
>> Basically, you'd use any varianter plugin on a job, and then you can
>> reuse the JSON generated on other jobs.
>>
>> TBH, I tweaked this one a bit.
>>
>>>> You should use some linter ;)
>>
>> I missed your point here... do you mean the style is not ideal?
> 
> As meant "pretty printer" :)
> 
> [
>   {
>     "paths": [
>       "/run/*"
>     ],
>     "variant": [
>       [
>         "/run/aarch64",
>         [
>           [
>             "/run/aarch64",
>             "arch",
>             "aarch64"
>           ]
>         ]
>       ]
>     ],
>     "variant_id": "aarch64"
>   },
>   ...
> 
> But since it is generated I'd rather generate it...

Actually, I think it's a good idea indeed to pretty print it, "python -m
json.tool" should do the job.

Thanks,
- Cleber.

> 
>>>
>>> Also, that's a long line, which will probably get longer as more support
>>> is added. Beyond 990 bytes, it starts risking problems with corruption
>>> over email. It's also hard to view what changes incrementally if the
>>> single line changes. Is there a way to pretty-print things across
>>> multiple lines, for shorter lines and easier reading of future diffs?
>>>
>>
>> Yes, good point.  I'll pretty print it.
>>
>> Just a disclaimer: I've chosen to use a JSON variants because it's a
>> core Avocado feature (doesn't require any external plugin), and the
>> results are 100% reproducible (the variants are static).  In the future,
>> we may consider also shipping (and depending) on other variants.
>>
>> One idea that is being maturing (and prototype) is a native QEMU
>> varianter.  There's some info here:
>>
>> https://trello.com/c/qW4kMw50/32-guest-abi-machine-type-cpu-model-test-cases
>>
>> Regards,
>> - Cleber.
>>