Re: [PATCH v4 1/6] tests/requirements.txt: bump up avocado-framework version to 101.0

[Philippe Mathieu-Daudé]

On 21/4/23 06:09, Cleber Rosa wrote:
> On 4/20/23 12:54, John Snow wrote:
> > On Thu, Apr 20, 2023, 5:19 AM Philippe Mathieu-Daudé 
> > <philmd@linaro.org> wrote:
> >
> >     On 17/4/23 19:44, Alex Bennée wrote:
> >     >
> >     > John Snow <jsnow@redhat.com> writes:
> >     >
> >     >> On Mon, Apr 17, 2023 at 9:43 AM Alex Bennée
> >     <alex.bennee@linaro.org> wrote:

> >     >>> diff --git a/tests/requirements.txt b/tests/requirements.txt
> >     >>> index 0ba561b6bd..a6f73da681 100644
> >     >>> --- a/tests/requirements.txt
> >     >>> +++ b/tests/requirements.txt
> >     >>> @@ -2,5 +2,5 @@
> >     >>>   # in the tests/venv Python virtual environment. For more info,
> >     >>>   # refer to: https://pip.pypa.io/en/stable/user_guide/#id1
> >     >>>   # Note that qemu.git/python/ is always implicitly installed.
> >     >>> -avocado-framework==88.1
> >     >>> +avocado-framework==101.0
> >     >>>   pycdlib==1.11.0
> >     >>> --
> >     >>> 2.39.2
> >     >>>
> >     >>
> >     >> I thought there were test failures that prohibited us from
> >     bumping the
> >     >> Avocado-Framework version. Did those get rectified recently?
> >
> >     No, still not working on Darwin. At this point I'm thinking at 
> > sending
> >     a patch clarifying testing QEMU with Avocado is not supported on
> >     Darwin.
> >
> >
> > What's broken on Darwin?
> Hi John,
>
> IIRC, the main issue is that the legacy runner (the one active in 88.1) 
> pickles some stuff that do not play nice under Darwin.  It has never (to 
> the best of my knowledge) worked under Darwin.

Correct.

> > Is it broken with avocado-framework==88.1 too? (Is this a regression?)
> It was already broken with avocado-framework==88.1,

Correct.

> but it should mostly 
> work with 101.0 (see previous reply).
>
> > We might need to consider entirely separate issues:
> >
> > (1) We need to upgrade avocado-framework to a supported LTS version >= 
> > v90 for dependency harmony with qemu.git/python testing
> >
> > (2) We need to upgrade avocado-framework to >= v101.0 to fix the 
> > stated checksum issue in this patch
> >
> > (3) We would like avocado tests to work on Darwin. (Have they ever 
> > worked? When did they break? etc.)
> >
> > My understanding is that this patch helps with 1 and 2 but not 3, but 
> > if Darwin support isn't a regression, this is still a step forward.
> To the best of my knowledge, this patch would help with 1, 2 and even 3.

As of today using v8.0.0:

$ make check-avocado
  GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 
tests/fp/berkeley-softfloat-3 dtc
  MKDIR   /Users/philmd/source/qemu/build/tests/results
  AVOCADO Downloading avocado tests VM image for aarch64
The image was downloaded:
Provider Version Architecture File
fedora   31      aarch64 
/Users/philmd/avocado/data/cache/by_location/4f156e531446a679cbfe13caef8b7c9f9f79aafa/Fedora-Cloud-Base-31-1.9.aarch64.qcow2
  AVOCADO tests/avocado
Fetching asset from 
tests/avocado/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_atf
Fetching asset from 
tests/avocado/boot_xen.py:BootXen.test_arm64_xen_411_and_dom0
Fetching asset from 
tests/avocado/boot_xen.py:BootXen.test_arm64_xen_414_and_dom0
Fetching asset from 
tests/avocado/boot_xen.py:BootXen.test_arm64_xen_415_and_dom0
Fetching asset from 
tests/avocado/machine_aarch64_virt.py:Aarch64VirtMachine.test_alpine_virt_tcg_gic_max
Fetching asset from 
tests/avocado/replay_kernel.py:ReplayKernelNormal.test_aarch64_virt
Fetching asset from 
tests/avocado/reverse_debugging.py:ReverseDebugging_AArch64.test_aarch64_virt
JOB ID     : 142a0b2d3960c7daccd657781d52f814e1617392
JOB LOG    : 
/Users/philmd/source/qemu/build/tests/results/job-2023-04-21T11.35-142a0b2/job.log

Avocado crashed: TypeError: cannot pickle '_thread.RLock' object
Traceback (most recent call last):

  File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/core/job.py", 
line 632, in run_tests
    summary |= suite.run(self)

  File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/core/suite.py", 
line 311, in run
    return self.runner.run_suite(job, self)

  File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/plugins/runner.py", 
line 390, in run_suite
    if not self.run_test(job, test_factory, queue, summary,

  File 
"/Users/philmd/source/qemu/build/tests/venv/lib/python3.10/site-packages/avocado/plugins/runner.py", 
line 170, in run_test
    proc.start()

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/process.py", 
line 121, in start
    self._popen = self._Popen(self)

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", 
line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/context.py", 
line 288, in _Popen
    return Popen(process_obj)

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", 
line 32, in __init__
    super().__init__(process_obj)

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_fork.py", 
line 19, in __init__
    self._launch(process_obj)

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/popen_spawn_posix.py", 
line 47, in _launch
    reduction.dump(process_obj, fp)

  File 
"/opt/homebrew/Cellar/python@3.10/3.10.9/Frameworks/Python.framework/Versions/3.10/lib/python3.10/multiprocessing/reduction.py", 
line 60, in dump
    ForkingPickler(file, protocol).dump(obj)

TypeError: cannot pickle '_thread.RLock' object

Please include the traceback info and command line used on your bug report
Report bugs visiting https://github.com/avocado-framework/avocado/issues/new
make: *** [check-avocado] Error 4

This is https://github.com/avocado-framework/avocado/issues/4888
which is still opened.

Manually using "avocado run --test-runner=nrunner" helps running the
tests but the console output is never displayed (see [*]) and once
finished Avocado lets various QEMU processes running in detached
mode, wasting resources.

[*] 
https://lore.kernel.org/qemu-devel/1d0baf8b-c757-265c-b206-07ca3f218b2a@linaro.org/