Re: Why we should avoid new submodules if possible

[Thomas Huth]

On 28/06/2022 12.30, Michael S. Tsirkin wrote:
> On Tue, Jun 28, 2022 at 12:21:39PM +0200, Thomas Huth wrote:
> > On 28/06/2022 12.03, Michael S. Tsirkin wrote:
> > [...]
> > > For biosbits if we are going this route then I feel a submodule is much
> > > better.  It records which version exactly each qemu version wants.
> >
> > As far as I know, you can also specify the version when using pip, can't
> > you? So that's not really an advantage here.
>
> But of course if you do you do not get updates ;) You do
> however rely on a 3rd party to faithfully provide you
> correct code based on the version, and host it forever.
>
> > On the contrary, submodules have a couple of disadvantages that I really
> > dislike:
> >
> > - submodules do not get updated automatically when doing a "git checkout",
> > we have to update them via a script instead. This causes e.g. trouble if you
> > rsync your source tree to a machine that has no access to the internet and
> > you forgot to update the submodule before the sync
>
> how is pip better?

You don't end up with an inconsistent source tree in that case (which 
happens with submodules quite easily, at least for me it happened a couple 
of times already). Either the machine has an internet connection, so that 
pip can install the missing bits, or it does not and the test has to be 
skipped. But if I copy the wrong state of a submodule around, things get 
messed up quite easily in my experience. Ok, you could say that this is just 
my special setup with rsync, but already given the fact that "git checkout" 
creates an inconsistent state of your source tree until you run the script 
for updating the submodules the next time is an indication that submodules 
are rather a shaky thing (e.g. if you'd create a tarball for somebody else 
from your source tree right after doing a "git checkout").

> > - the content of submodules is not added to the tarballs that get created on
> > the git forges automatically. There were lots of requests from users in the
> > past that tried to download a tarball from github and then wondered why they
> > couldn't compile QEMU.
>
> how is pip better here?

You don't get incomplete/non-working tarballs in that case.

> > - we include the submodule content in our release tarballs, so people get
> > the impression that hte submodule content is part of the QEMU sources. This
> > has two disadvantages:
> >   * We already got bug reports for the code in the submodule,
> >     where people did not understand that they should report that
> >     rather to the original project instead (i.e. you ship it - you
> >     own it)
> >   * People get the impression that QEMU is a huge monster
> >     application if they count the number of code lines, run
> >     their code scanner tools on the tarball contents, etc.
> >     Remember "nemu", for example, where one of the main complaints
> >     was that QEMU has too many lines of code?
>
> I think we can skip the checkout in the tarball if we like.
> If people want to run the test they can checkout then.

Release tarballs don't include the ".git" folder infrastructur, so everybody 
who downloads a tarball will simply never be able to run the test.

> > - If programs includes code via submodules, this gets a higher
> >    burder for distro maintainers, since they have to patch each
> >    and every package when there is a bug, instead of being able to
> >    fix it in one central place.
>
> Come on, this is just a test. We *really* don't care if an ISO
> we use to test ACPI is using an exploitable version of grub.

Wait, I thought we were only talking about tappy here? The ISO binaries 
should certainly *not* be bundled in the QEMU tarballs (they are too big 
already anyway, we should rather think of moving the firmware binaries out 
of the tarball instead).

 Thomas