Re: [PATCH 0/3] scripts/make-release: Decrease the size of the release tarballs

[Daniel P . Berrangé]

On Mon, Jul 04, 2022 at 08:42:51AM +0200, Thomas Huth wrote:
> Our release tarballs are huge - qemu-7.0.0.tar.xz has a size of 119 MiB.
> If you look at the contents, more than half of the size is used for the
> edk2 sources that we ship along to provide the sources for the firmware
> binaries, too. This feels very wrong, why do we urge users to download
> such huge tarballs while 99.9% of them never will rebuilt the firmware
> sources? We were also struggeling a bit in the past already with server
> load and costs, so we should really try to decrease the size of our
> release tarballs to a saner level.
> 
> Fortunately, edk2 has a permissive BSD license, so we are not forced
> to distribute the sources for this. Thus instead of packaging the whole
> edk2 source tree in our tarballs, let's just do the bare minimum and
> provide the license information and a pointer to where the users can
> download the edk2 sources instead. This decreases the size of our tarballs
> already to the half of the original size.

Regardless of license, we are not required to bundle the source code
and binaries in the same tarball. We've merely done that because it
was a convenient & easy way to approach the problem historically.

It would be valid to not ship *any* of the source for the pre-built
roms in the main qemu-x.y.z.tar.xz file /provided/ we ensure that
we *always* have a qemu-firmware-src-x.y.z.tar.xz file alongside it.


> Some few additional MiBs can be saved by omitting the sources of the
> skiboot firmware, which has a permissive license, too (see second patch).
> The final patch is rather cosmetics only - it drops some additional
> .yml and .git files from the tarball that are of no use for the normal
> user without the corresponding git repository.

I look at a few more scenarios

  * Current tarball:            119 MB
  * minus edk/skiboot source:    54 MB
  * also minus edk2 binaries:    45 MB
  * also minus pc-bios/ + roms/: 19 MB
  * minus roms/ only:            31 MB

IOW, cutting the tarball in half is great, but if we split off firmware
binaries and source into completely separated tarballs we would win big.
If we fully split off only the firmware source we still win quite alot.

IOW, rather than special casing edk/skiboot, I would prefer to see us
have make a consistent approach to firmware.

Either

 * qemu-x.y.z.tar.gz              (only qemu maintained src, 19 MB)
 * qemu-firmware-x.y.z.tar.gz     (pre-built blobs aka pc-bios/, 13 MB)
 * qemu-firmware-src-x.y.z.tar.gz (source for pre-built blobs aka roms/, 92 MB)

Or 

 * qemu-x.y.z.tar.gz              (qemu maintained src and pre-built blobs, 31 
MB)
 * qemu-firmware-src-x.y.z.tar.gz (source for pre-built blobs aka roms/, 92 MB)


The second option is probably the least disruptive option for end users
building QEMU directly, while still giving distros most of the benefits
they desire. And probably easiest to put into practice for us.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|