Re: Building some targets serially, others in parallel

[Renaud Pacalet]

On 22/08/2020 22:46, dan soucy wrote:
> Hi list.
> 
> I find myself in a situation where I want the makefile to know not to build
> certain targets in parallel, even though they do not actually depend on
> each
> other.
> To explain:
> Suppose we have three executables to build: `server`, `client`, `demo`.
> Each
> of these has an associated source file.
> 
> There are many other source files -- maybe a dozen. They will be
> compiled to
> object files which are depended on by all three executable files.
> 
> Normally, the quick solution is to add one target for each object file
> and tell `make` to invoke the compiler on each in a separate process,
> using as many processors as are available.
> 
> But -- in this case, the design of the compiler is such that telling the
> compiler to build all required object files in parallel is much faster than
> invoking the compiler many times. This is due to a (relatively) slow
> start-up and also the re-use of data between source files (such as types).
> 
> So instead, we have something like this:
> 
> ```
> server client demo: $(OBJECTS)
>     $(COMPILER) -j 4 $@ -o $< ```
> 
> Running `make -j 4 server client demo` with the object files not built
> will result in the compiler being invoked thrice, and each will begin by
> compiling the same object files.

I do not understand why the compiler would "begin by compiling the same
object files". Normally, one of the main make features is to avoid doing
useless things. If the same source file gets compiled several times
there must be a problem with your Makefile. For instance, instead of
having rules where the object file itself is the target, you have rules
where the target is something else.

Moreover, your recipe looks a bit strange: the way you use automatic
variables suggests that you are compiling the target ($@) and output the
first prerequisite ($<). Are you sure your recipe is correct?

> The result is that this actually takes
> longer than running `make` serially.
> 
> However, if the object files are already built, then `server`, `client`,
> and
> `demo` can be built faster in parallel.
> 
> For the curious, the compiler is GHC and the code is Haskell.

GNU make 4.3 introduces the grouped targets (`&:` separator). So, if you
want to build all your object files with one single execution of a
recipe, and if you use GNU make 4.3, you could try:

$(OBJECTS) &: $(SOURCES)
    $(COMPILER) -j4 $^ ...

This tells make that all object files are built by a single execution of
the recipe. One last improvement would be to build OBJECTS (and SOURCES)
such that they contain only outdated object files.
-- 
Renaud Pacalet
Télécom Paris
Campus SophiaTech
450 Route des Chappes, CS 50193
06904 Biot Sophia Antipolis cedex, FRANCE
Tel : +33 (0) 4 9300 8402
Web : http://www.telecom-paris.fr/