Re: [PATCH v6 00/10] Introduce new acpi/smbios avocado tests using biosbits

[Michael S. Tsirkin]

On Sat, Oct 22, 2022 at 06:28:32AM +0530, Ani Sinha wrote:
> 
> 
> On Fri, Oct 21, 2022 at 21:32 Alex Bennée <alex.bennee@linaro.org> wrote:
> 
> 
>     Ani Sinha <ani@anisinha.ca> writes:
> 
>     > On Fri, 21 Oct, 2022, 5:52 pm Ani Sinha, <ani@anisinha.ca> wrote:
>     >
>     >  On Fri, 21 Oct, 2022, 5:26 pm Alex Bennée, <alex.bennee@linaro.org>
>     wrote:
>     >
>     >  Ani Sinha <ani@anisinha.ca> writes:
>     >
>     >  > On Fri, Oct 21, 2022 at 3:10 PM Michael S. Tsirkin <mst@redhat.com>
>     wrote:
>     >  >>
>     >  >> On Fri, Oct 21, 2022 at 10:30:09AM +0100, Alex Bennée wrote:
>     >  >> >
>     >  >> > Ani Sinha <ani@anisinha.ca> writes:
>     >  >> >
>     >  >> > > On Fri, Oct 21, 2022 at 2:02 PM Michael S. Tsirkin <
>     mst@redhat.com> wrote:
>     >  >> > >>
>     >  >> > >> On Fri, Oct 21, 2022 at 05:45:15AM +0530, Ani Sinha wrote:
>     >  >> > >> > And have multiple platform specific branches in bits that 
> have
>     fixes for those
>     >  >> > >> > platforms so that bits can run there. Plus the existing test
>     can be enhanced to
>     >  >> > >> > pull in binaries from those branches based on the platform on
>     which it is being
>     >  >> > >> > run.
>     >  >> > >> >
>     >  >> > >>
>     >  >> > >> What a mess.
>     >  >> > >> Who is going to be testing all these million platforms?
>     >  >> > >
>     >  >> > > I am not talking about branches in QEMU but branches in bits.
>     >  >> > > If you are going to test multiple platforms, you do need to 
> build
>     bits
>     >  >> > > binaries for them. There is no way around it.
>     >  >> > > bits is not all platform independent python. It does have binary
>     executables.
>     >  >> > >
>     >  >> > > Currently bits is built only for the x86 platform. Other
>     platforms are
>     >  >> > > not tested. I doubt if anyone even tried building bits for arm 
> or
>     >  >> > > mips.
>     >  >> >
>     >  >> > I'm not worried about test bits on other targets, but we do run 
> x86
>     >  >> > targets on a number of hosts. The current reliance on a special
>     patched
>     >  >> > host build tool for only one architecture is the problem. If  we
>     just
>     >  >> > download the iso that problem goes away.
>     >  >>
>     >  >> 👍what he said.
>     >  >
>     >  > Yes, in that case the problem is that upstream bits does not pass all
>     >  > the test out of the box. Hence we are taking this approach of keeping
>     >  > some test scripts in QEMU repo and modifying them. Then generating 
> the
>     >  > iso with the modified scripts. It also helps developers who want to
>     >  > write new tests or make enhancements to existing tests.
>     >  > If modifications need to be made to tests, they need to be versioned.
>     >  > We have gone through the route of not using submodules and I am not
>     >  > going to open that can of worms again.
>     >
>     >  We have added a mirror of biosbits to the QEMU project so there is no
>     >  reason why we can't track changes and modifications there (we do this
>     >  for TestFloat which is forked from the upstream SoftFloat code).
> 
> 
> One last option. Commit this patch set but also double commit patch 3 to the
> bits repo so that we can build an iso that would successfully run all tests 
> for
> a separate platform independent test to be written later.
> 
> Then we will have two tests:
> 
> - this one for developers writing new test.
> - platform independent one for a basic sanity.
> 
> I’m just documenting the fact that I have proposed ideas that can work where
> all can be happy. It’s up to others to take it or keep objecting and killing
> motivations for freelance contributors. 

I think it's ok to apply this as is for starters.
Anyone has objections?

Down the road I think things should be refactored slightly to work as follows:
- test developers can check out biosbits repo to create the iso
- everyone else gets iso downloaded

Objections to this plan?


> 
> 
> 
>     >
>     > 
>