Re: [Qemu-devel] converting build system to Meson?

[Daniel P . Berrangé]

On Wed, Mar 06, 2019 at 07:50:49PM +0100, Marc-André Lureau wrote:
> Hi Paolo
> 
> On Wed, Mar 6, 2019 at 7:12 PM Paolo Bonzini <address@hidden> wrote:
> >
> > Hi all,
> >
> > lately I have been thinking of converting the QEMU build system to
> > Meson.  Meson is a relatively new build system that can replace
> > Autotools or hand-written Makefiles such as QEMU; as a die-hard
> > Autotools fan, I must say that Meson is by far better than anything else
> > that has ever tried to replace Autotools, and actually has the potential
> > to do so.
> >
> > Advantages of Meson that directly matter for QEMU include:
> >
> > - build definitions in a very readable and user friendly DSL, which
> > supports looping and conditions.
> >
> > - ability to introspect the build definitions so that you can find out
> > what is built without building it (the DSL is not Turing complete and
> > most objects in it are immutable, so it cannot be abused that much :))
> >
> > - support for a non-recursive build from per-subdirectory input (similar
> > to Makefile.objs)
> >
> > - ease of distributing a full copy of Meson to support distros that ship
> > an older version (no dependencies apart from Python 3.5).  At 40000
> > lines of Python, Meson is relatively small.
> >
> >
> > Unlike Autoconf, meson.build files include both feature detection and
> > build rules.  Also unlike Autoconf, Meson must be present on the system
> > of whoever builds QEMU, which is why the last bullet above matters.
> > However, it is possible to keep a configure script, if only to provide
> > backwards command line compatibility and support a bundled copy of Meson.
> 
> Yes, we will probably have a transition period where both build-sys
> have to co-exist (same story with libslirp, which I proposed as
> meson-only first)

I'd very much *not* want to have them co-existing. Doing that significantly
increases the maint burden for any developer needing to change build rules.
It also introduces significant risks that each build system produces
different results. GTK took this approach and created a number of serious
problems where meson results differed from autoconf results.

> > Of course a full conversion of QEMU's build system, including the 7k
> > lines configure script is not easy.  In addition, even though Meson
> 
> I don't think a complete transition in one go is feasible. Could we
> start with some basic things?
> 
> For example, we could quite easily start converting tools (qemu-img
> etc), some tests...

The difficulty of doing this is that alot of our code is shared between
the tools and the system emulators.  It is particularly problematic for
code that is auto-generated upfront. For example trace files and qapi
files. The new build system would have to be able to consume .o & .c
files already built by the other build system, with correct dependancy
ordering to ensure parallel build succeeds.  I'm not saying it is
impossible, but it is certainly very hard, to the extent that it may
well be easier to do a big-bang.


> > even just "include" the Meson-generated build system into Make and keep
> > a (progressively more) minimal Makefile veneer around it.  I benchmarked
> > Make a bit and its slowness of QEMU on a do-nothing build is simply due
> > to the complexity of the unnest-vars macro machinery; Make could parse a
> > synthetic 90000-target Makefile in 2 seconds (QEMU has 9000 object
> > files), and the ninja->Make converter is careful to avoid traps where
> > Make has quadratic complexity.
> >
> >
> > To be clear, this is not something I am going to work anytime soon.
> > Still in my opinion it's a worthwhile exercise to think about it.  As I
> > will see in the next few weeks what the Meson maintainers' reaction will
> > be to some of the proposed extensions, I wanted to gauge the reactions
> > of you all as well. :)
> 
> I like meson better than anything else. It is still young, one can
> find quickly limitations & documentation could be better (if you
> compare it with make/autoconf/automake reference manual for ex).

I actually though the meson documentation was really pretty decent.
Where I've personally had problems was when trying to find docs for
features that don't exist yet in meson :-P

I like it better than any of the other programs that has been put
forwards as an autotools killer, to the extent I've been converting
all my personal projects to it. It might not be mature enough to
cover all the hard bits, but what it does do, it does very clearly
making the resulting build files nice to maintain.

Converting QEMU though would certainly be a large challenge in scope
pushing the boundaries of what's possible.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|