Re: [RFC PATCH-for-5.2] tests/acceptance: Disable Spartan-3A DSP 1800A test

[Philippe Mathieu-Daudé]

On 11/9/20 10:16 AM, Thomas Huth wrote:
> On 09/11/2020 10.09, Philippe Mathieu-Daudé wrote:
>> On 11/9/20 10:04 AM, Thomas Huth wrote:
>>> On 09/11/2020 09.07, Philippe Mathieu-Daudé wrote:
>>>> This test is regularly failing on CI :( Do not run it automatically.
>>>>
>>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>>> ---
>>>>  tests/acceptance/boot_linux_console.py | 1 +
>>>>  tests/acceptance/replay_kernel.py      | 1 +
>>>>  2 files changed, 2 insertions(+)
>>>>
>>>> diff --git a/tests/acceptance/boot_linux_console.py 
>>>> b/tests/acceptance/boot_linux_console.py
>>>> index 8f433a67f84..010e8790c0f 100644
>>>> --- a/tests/acceptance/boot_linux_console.py
>>>> +++ b/tests/acceptance/boot_linux_console.py
>>>> @@ -1025,6 +1025,7 @@ def test_m68k_mcf5208evb(self):
>>>>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>>>>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>>>>  
>>>> +    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might 
>>>> timeout')
>>>>      def test_microblaze_s3adsp1800(self):
>>>>          """
>>>>          :avocado: tags=arch:microblaze
>>>> diff --git a/tests/acceptance/replay_kernel.py 
>>>> b/tests/acceptance/replay_kernel.py
>>>> index 00c228382bd..c1f5fa4de71 100644
>>>> --- a/tests/acceptance/replay_kernel.py
>>>> +++ b/tests/acceptance/replay_kernel.py
>>>> @@ -280,6 +280,7 @@ def test_m68k_mcf5208evb(self):
>>>>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>>>>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>>>>  
>>>> +    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might 
>>>> timeout')
>>>>      def test_microblaze_s3adsp1800(self):
>>>>          """
>>>>          :avocado: tags=arch:microblaze
>>>
>>> I think this is a recent regression - it hasn't been failing in the past. We
>>> should first try to find out why it is failing now before sending it to the
>>> @skipUnless nirvana... could you maybe add it to the "Known issues" at
>>> https://wiki.qemu.org/Planning/5.2 instead?
>>
>> I agree it looks like a regression.
>>
>> I disagree we should keep broken tests failing the pipeline,
>> even if we are not using a Gating CI.
> 
> But what happens if you disable the test at this point in time now? I think
> nobody is going to look into this issue anymore since nobody feels
> responsible. Thus the bug simply get completely ignored.

Corollary: Nobody will use or trust GitLab CI...

> Please add it at
> least the the "Known issues" section.

Will do.