[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: biosbits test failing on origin/master
From: |
Daniel P . Berrangé |
Subject: |
Re: biosbits test failing on origin/master |
Date: |
Wed, 16 Nov 2022 09:21:14 +0000 |
User-agent: |
Mutt/2.2.7 (2022-08-07) |
On Wed, Nov 16, 2022 at 08:34:00AM +0530, Ani Sinha wrote:
> On Wed, Nov 16, 2022 at 12:18 AM John Snow <jsnow@redhat.com> wrote:
> >
> > On Tue, Nov 15, 2022 at 9:31 AM Ani Sinha <ani@anisinha.ca> wrote:
> > >
> > > On Tue, Nov 15, 2022 at 3:36 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > >
> > > > On Tue, Nov 15, 2022 at 9:07 AM Ani Sinha <ani@anisinha.ca> wrote:
> > > > >
> > > > > On Tue, Nov 15, 2022 at 5:13 AM John Snow <jsnow@redhat.com> wrote:
> > > > > >
> > > > > > On Thu, Nov 10, 2022 at 11:22 PM Ani Sinha <ani@anisinha.ca> wrote:
> > > > > > >
> > > > > > > On Thu, Nov 10, 2022 at 11:37 PM John Snow <jsnow@redhat.com>
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > Hiya, on today's origin/master
> > > > > > > > (2ccad61746ca7de5dd3e25146062264387e43bd4) I'm finding that
> > > > > > > > "make
> > > > > > > > check-avocado" is failing on the new biosbits test on my local
> > > > > > > > development machine:
> > > > > > > >
> > > > > > > > (001/193)
> > > > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > > FAIL: True is not false : The VM seems to have failed to
> > > > > > > > shutdown in
> > > > > > > > time (83.65 s)
> > > > > > > >
> > > > > > > > Is this a known issue, or should I begin to investigate it?
> > > > > > >
> > > > > > > In my test environment it does pass.
> > > > > > >
> > > > > > > $ ./tests/venv/bin/avocado run -t acpi tests/avocado
> > > > > > > Fetching asset from
> > > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits
> > > > > > > JOB ID : 35726df7d3c2e0f41847822620c78195ba45b9b9
> > > > > > > JOB LOG :
> > > > > > > /home/anisinha/avocado/job-results/job-2022-11-11T09.42-35726df/job.log
> > > > > > > (1/1)
> > > > > > > tests/avocado/acpi-bits.py:AcpiBitsTest.test_acpi_smbios_bits:
> > > > > > > PASS (57.57 s)
> > > > > > > RESULTS : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 |
> > > > > > > INTERRUPT 0
> > > > > > > | CANCEL 0
> > > > > > > JOB TIME : 63.82 s
> > > > > > >
> > > > > > > However, I have seen that on certain slower test machines or when
> > > > > > > run
> > > > > > > within a virtual machine, the test can take longer to complete
> > > > > > > and 60
> > > > > > > secs may not always be enough. In those cases raising the maximum
> > > > > > > completion time to 90 secs helps. Perhaps you can try this and
> > > > > > > let me
> > > > > > > know if it helps:
> > > > > >
> > > > > > Hmm - I'm running on a fairly modern machine and not in a VM. Do you
> > > > > > have an invocation to share that exists outside of the avocado
> > > > > > machinery
> > > > >
> > > > > If you pass V=1 in the environment then it dumps the QEMU command line
> > > > > that was used to run the test. You also need to comment out the line
> > > > > > shutil.rmtree(self._workDir)
> > > > > in tearDown() so that the iso is not cleaned up.
> > > >
> > > > Maybe I will send out this patch once we have figured out what is
> > > > going on with your environment:
> > > > https://gitlab.com/anisinha/qemu/-/commit/5e8c629fdecc7cb650e4acaad8a8fcc2b248434e
> > > >
> > > > I ran the test on another box sitting in my office running centos7.9
> > > > and it passed as well.
> > >
> > > For the records,
> > > I ran make check-avocado on my Ubuntu 222.04 laptop:
> > > https://pastebin.com/0ZKEEQds
> > > On a separate centos 7.9 box (fairly new) : https://pastebin.com/QWLGDbp4
> >
> > As a question: Is it necessary to implement your own timeout here?
> > What's wrong with relying on Avocado's timeout?
>
> When I wrote the test I was not aware of the avocado timeout. Hence
> implemented mine. However, I kind of think that timing out from the
> test itself rather than from the framework provides an opportunity to
> provide more meaningful information to the user when the timeout
> happens. Of Course we can improve upon the current "FAIL: True is not
> false" assertion message.
>
> > My hunch is that you'll get greater flexibility by leaning into the
> > tool suite's configuration instead of hardcoding your own...
> >
> > For what it's worth, I am now trying to run this test manually by doing:
> >
> > > time ./qemu-system-x86_64 -cdrom
> > > /var/tmp/acpi-bits-b_br0ch8.tmp/bits-2020.iso -icount auto
> >
> > This is not a quick test:
> >
> > ________________________________________________________
> > Executed in 86.50 secs fish external
> > usr time 86.57 secs 0.00 micros 86.57 secs
> > sys time 0.30 secs 903.00 micros 0.29 secs
> >
> >
> > This isn't the *most* cutting edge machine, but it's a Intel(R)
> > Core(TM) i7-9850H CPU @ 2.60GHz with 32GB of memory and NVME storage.
> > Is your machine really passing this test in under 60 seconds flat?
>
> Yes it is both on my 3 year old lenovo laptop with 16 Gib memory:
Note gitlab.com public CI shared runners are *very* small VM instances.
IIUC, currently we can expect the runner to have as little as 1 vCPU,
and 3.75 GB of RAM.
With regards,
Daniel
--
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o- https://fstop138.berrange.com :|
|: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
- Re: biosbits test failing on origin/master, (continued)
- Re: biosbits test failing on origin/master, Ani Sinha, 2022/11/11
- Re: biosbits test failing on origin/master, John Snow, 2022/11/14
- Re: biosbits test failing on origin/master, Ani Sinha, 2022/11/14
- Re: biosbits test failing on origin/master, Ani Sinha, 2022/11/15
- Re: biosbits test failing on origin/master, Ani Sinha, 2022/11/15
- Re: biosbits test failing on origin/master, John Snow, 2022/11/15
- Re: biosbits test failing on origin/master, John Snow, 2022/11/15
- Re: biosbits test failing on origin/master, Ani Sinha, 2022/11/15
- Re: biosbits test failing on origin/master, Ani Sinha, 2022/11/15
- Re: biosbits test failing on origin/master,
Daniel P . Berrangé <=