Re: A build system using ``Recursive Make Considered harmful''

[Scott A Crosby]

On 11 Aug 2001, Tom Tromey wrote:

> Scott> Yep.. I thought I'd put up my results and assessments.
> Scott> Parallel/distributed building will likely become a more and
> Scott> more valuable thing in the future, and its something that is
> Scott> very hard to do without this.
>
> Automake already supports parallel builds.  It has for years (I don't
> think there was ever a public release that didn't).  Now, in some
> situations parallelism is limited because we serialize across
> directories.  I doubt that is a very big problem, at least with the
> newer versions of GNU make (older versions didn't allow sub-makes to
> parallelize).

Nope, the moment you force any serialization by module or directory on the
dag, you have the potential of breaking things.

Because make doesn't know in which order you must build the directories,
if you hardcode it in, you can still cause things to break. This is what I
tend to encounter when trying to parallel build programs that use
recursive makes. So although automake itself may support parallel builds
in that case, its not somethng one can really rely on.

Its even worse when files in two directories depend on files in the other
directory. I have examples of this where I have testing programs in two
directories that depend on libraries in both directories being built.

My build system, assuming correct dependencies, (which are computed
automatically except for building libraries and binaries), cannot have
these flaws. 'make -j' and 'make' both work.

The fact that it may also be faster (hard to benchmark) is not as relevant
the fact that make catches all bugs and always knows exactly what needs
rebuilding.


Go read the origional paper I reference. Recursive makes are fragile, and
even more so in parallel building. Automake does recursive makes. I avoid
that.


Scott