Re: [PATCH] tests/acceptance: Use imgtec.com URL for Fedora 22 artifacts

[Willian Rampazzo]

Hi Phil,

On Tue, Mar 2, 2021 at 12:45 PM Philippe Mathieu-Daudé
<philmd@redhat.com> wrote:
>
> On 3/2/21 4:27 PM, Willian Rampazzo wrote:
> > Hi Phil,
> >
> > On Mon, Mar 1, 2021 at 11:11 AM Philippe Mathieu-Daudé <f4bug@amsat.org> 
> > wrote:
> >>
> >> On 3/1/21 2:30 PM, Wainer dos Santos Moschetta wrote:
> >>> Hi,
> >>>
> >>> On 2/28/21 7:43 PM, Philippe Mathieu-Daudé wrote:
> >>>> Having artifacts stored in personal namespace is not ideal,
> >>>> as these might get closed and disappear. Use the original URL
> >>>> where these artifacts could be found.
> >>>>
> >>>> For more references:
> >>>> https://fedoraproject.org/wiki/Architectures/MIPS/2015Bootstrap/mips64el
> >>>>
> >>>> Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> >>>> ---
> >>>>   tests/acceptance/boot_linux_console.py | 4 ++--
> >>>>   tests/acceptance/replay_kernel.py      | 4 ++--
> >>>>   2 files changed, 4 insertions(+), 4 deletions(-)
> >>>>
> >>>> diff --git a/tests/acceptance/boot_linux_console.py
> >>>> b/tests/acceptance/boot_linux_console.py
> >>>> index eb012867997..138bd8cb7d3 100644
> >>>> --- a/tests/acceptance/boot_linux_console.py
> >>>> +++ b/tests/acceptance/boot_linux_console.py
> >>>> @@ -239,8 +239,8 @@ def test_mips64el_malta_5KEc_cpio(self):
> >>>>           :avocado: tags=machine:malta
> >>>>           :avocado: tags=endian:little
> >>>>           """
> >>>> -        kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
> >>>> -                      'raw/9ad2df38/mips/malta/mips64el/'
> >>>> +        kernel_url = ('http://mipsfedora.imgtec.com/development/22/'
> >>>> +                      'mips64el/images/20150601/'
> >>>>                         'vmlinux-3.19.3.mtoman.20150408')
> >>>
> >>> http://mipsfedora.imgtec.com/development/22/mips64el/images/20150601/vmlinux-3.19.3.mtoman.20150408
> >>> did not get resolve here. Could you please double-check the url is 
> >>> correct?
> >>
> >> Yes it is correct, the domain is dead. It was working when I wrote
> >> the test. Should I remove the test instead?
> >>
> >
> > This test is already skipped by default because of the
> > AVOCADO_ALLOW_UNTRUSTED_CODE flag.
>
> QEMU's .gitlab-ci.yml does (see acceptance_definition):
>
>   before_script:
>     - mkdir -p ~/.config/avocado
>     - echo "[datadir.paths]" > ~/.config/avocado/avocado.conf
>     - echo "cache_dirs = ['${CI_PROJECT_DIR}/avocado-cache']"
>            >> ~/.config/avocado/avocado.conf
>     - echo -e '[job.output.testlogs]\nstatuses = ["FAIL", "INTERRUPT"]'
>            >> ~/.config/avocado/avocado.conf
>     - if [ -d ${CI_PROJECT_DIR}/avocado-cache ]; then
>         du -chs ${CI_PROJECT_DIR}/avocado-cache ;
>       fi
>     - export AVOCADO_ALLOW_UNTRUSTED_CODE=1
>

This is true when running the test in the CI. What happens if I try to
run this test locally? It will be skipped unless I set
AVOCADO_ALLOW_UNTRUSTED_CODE=1.

> > Even if the flag is set to allow
> > the test to run, it will be skipped because the cache lacks the asset.
>
> Same file, we save the cache between runs:
>
>   cache:
>     key: "${CI_JOB_NAME}-cache"
>     paths:
>       - ${CI_PROJECT_DIR}/avocado-cache
>

In the CI environment, the test will execute because the asset is
already cached. Again, what happens if we merge this code and I try to
run it locally? the test will be skipped because I don't have the
asset.

The QEMU GitLab CI won't be affected by an invalid URL and will run
the test until the cache is cleared. If someone tries to run the
pipeline on a new instance of a GitLab CI, the test will be skipped.

With my suggestion, developers will have the chance to run the test
locally if they have the correct file and register it to their local
cache manually.

> > Instead of removing the test, my suggestion is to add information to
> > allow people to run it locally, like:
> >
> > diff --git a/tests/acceptance/boot_linux_console.py
> > b/tests/acceptance/boot_linux_console.py
> > index eb01286799..23efc18a99 100644
> > --- a/tests/acceptance/boot_linux_console.py
> > +++ b/tests/acceptance/boot_linux_console.py
> > @@ -239,11 +239,13 @@ def test_mips64el_malta_5KEc_cpio(self):
> >          :avocado: tags=machine:malta
> >          :avocado: tags=endian:little
> >          """
> > -        kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
> > -                      'raw/9ad2df38/mips/malta/mips64el/'
> > -                      'vmlinux-3.19.3.mtoman.20150408')
> > +        # to run this test locally, download the following asset and 
> > register
> > +        # it manually to the avocado cache with:
> > +        #   $(BUILD_DIR)/tests/venv/bin/avocado assets register \
> > +        #   vmlinux-3.19.3.mtoman.20150408 <path to the file>
> > +        kernel_name = 'vmlinux-3.19.3.mtoman.20150408'
>
> In this particular case it could work because the name is not generic,
> but what about the generic names ("vmlinux" for example)?
>
> Can't we register assets using any string to hash?
>

Right now it is not supported, but I opened an issue to discuss it
here: https://github.com/avocado-framework/avocado/issues/4435

> >          kernel_hash = '00d1d268fb9f7d8beda1de6bebcc46e884d71754'
> > -        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> > +        kernel_path = self.fetch_asset(kernel_name, asset_hash=kernel_hash)
> >          initrd_url = ('https://github.com/groeck/linux-build-test/'
> >                        'raw/8584a59e/rootfs/'
> >                        'mipsel64/rootfs.mipsel64r1.cpio.gz')
> > diff --git a/tests/acceptance/replay_kernel.py
> > b/tests/acceptance/replay_kernel.py
> > index c1cb862468..180e915264 100644
> > --- a/tests/acceptance/replay_kernel.py
> > +++ b/tests/acceptance/replay_kernel.py
> > @@ -397,11 +397,13 @@ def test_mips64el_malta_5KEc_cpio(self):
> >          :avocado: tags=endian:little
> >          :avocado: tags=slowness:high
> >          """
> > -        kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
> > -                      'raw/9ad2df38/mips/malta/mips64el/'
> > -                      'vmlinux-3.19.3.mtoman.20150408')
> > +        # to run this test locally, download the following asset and 
> > register
> > +        # it manually to the avocado cache with:
> > +        #   $(BUILD_DIR)/tests/venv/bin/avocado assets register \
> > +        #   vmlinux-3.19.3.mtoman.20150408 <path to the file>
> > +        kernel_name = 'vmlinux-3.19.3.mtoman.20150408'
> >          kernel_hash = '00d1d268fb9f7d8beda1de6bebcc46e884d71754'
> > -        kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
> > +        kernel_path = self.fetch_asset(kernel_name, asset_hash=kernel_hash)
> >          initrd_url = ('https://github.com/groeck/linux-build-test/'
> >                        'raw/8584a59e/rootfs/'
> >                        'mipsel64/rootfs.mipsel64r1.cpio.gz')
> >
> > What do you think?
> >
> > ps. WARNING, above code not tested.
> >
> >>>>           kernel_hash = '00d1d268fb9f7d8beda1de6bebcc46e884d71754'
> >>>>           kernel_path = self.fetch_asset(kernel_url,
> >>>> asset_hash=kernel_hash)
> >>>> diff --git a/tests/acceptance/replay_kernel.py
> >>>> b/tests/acceptance/replay_kernel.py
> >>>> index c1cb8624683..efdd4233bc2 100644
> >>>> --- a/tests/acceptance/replay_kernel.py
> >>>> +++ b/tests/acceptance/replay_kernel.py
> >>>> @@ -397,8 +397,8 @@ def test_mips64el_malta_5KEc_cpio(self):
> >>>>           :avocado: tags=endian:little
> >>>>           :avocado: tags=slowness:high
> >>>>           """
> >>>> -        kernel_url = ('https://github.com/philmd/qemu-testing-blob/'
> >>>> -                      'raw/9ad2df38/mips/malta/mips64el/'
> >>>> +        kernel_url = ('http://mipsfedora.imgtec.com/development/22/'
> >>>> +                      'mips64el/images/20150601/'
> >>>>                         'vmlinux-3.19.3.mtoman.20150408')
> >>>>           kernel_hash = '00d1d268fb9f7d8beda1de6bebcc46e884d71754'
> >>>>           kernel_path = self.fetch_asset(kernel_url,
> >>>> asset_hash=kernel_hash)
> >>>
> >>
> >
>