Re: [RFC PATCH-for-5.2] tests/acceptance: Disable Spartan-3A DSP 1800A test

[Thomas Huth]

On 09/11/2020 10.09, Philippe Mathieu-Daudé wrote:
> On 11/9/20 10:04 AM, Thomas Huth wrote:
>> On 09/11/2020 09.07, Philippe Mathieu-Daudé wrote:
>>> This test is regularly failing on CI :( Do not run it automatically.
>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
>>> ---
>>>  tests/acceptance/boot_linux_console.py | 1 +
>>>  tests/acceptance/replay_kernel.py      | 1 +
>>>  2 files changed, 2 insertions(+)
>>>
>>> diff --git a/tests/acceptance/boot_linux_console.py 
>>> b/tests/acceptance/boot_linux_console.py
>>> index 8f433a67f84..010e8790c0f 100644
>>> --- a/tests/acceptance/boot_linux_console.py
>>> +++ b/tests/acceptance/boot_linux_console.py
>>> @@ -1025,6 +1025,7 @@ def test_m68k_mcf5208evb(self):
>>>          tar_hash = 'ac688fd00561a2b6ce1359f9ff6aa2b98c9a570c'
>>>          self.do_test_advcal_2018('07', tar_hash, 'sanity-clause.elf')
>>>  
>>> +    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might 
>>> timeout')
>>>      def test_microblaze_s3adsp1800(self):
>>>          """
>>>          :avocado: tags=arch:microblaze
>>> diff --git a/tests/acceptance/replay_kernel.py 
>>> b/tests/acceptance/replay_kernel.py
>>> index 00c228382bd..c1f5fa4de71 100644
>>> --- a/tests/acceptance/replay_kernel.py
>>> +++ b/tests/acceptance/replay_kernel.py
>>> @@ -280,6 +280,7 @@ def test_m68k_mcf5208evb(self):
>>>          file_path = self.fetch_asset(tar_url, asset_hash=tar_hash)
>>>          self.do_test_advcal_2018(file_path, 'sanity-clause.elf')
>>>  
>>> +    @skipUnless(os.getenv('AVOCADO_TIMEOUT_EXPECTED'), 'Test might 
>>> timeout')
>>>      def test_microblaze_s3adsp1800(self):
>>>          """
>>>          :avocado: tags=arch:microblaze
>>
>> I think this is a recent regression - it hasn't been failing in the past. We
>> should first try to find out why it is failing now before sending it to the
>> @skipUnless nirvana... could you maybe add it to the "Known issues" at
>> https://wiki.qemu.org/Planning/5.2 instead?
> 
> I agree it looks like a regression.
> 
> I disagree we should keep broken tests failing the pipeline,
> even if we are not using a Gating CI.

But what happens if you disable the test at this point in time now? I think
nobody is going to look into this issue anymore since nobody feels
responsible. Thus the bug simply get completely ignored. Please add it at
least the the "Known issues" section.

 Thomas