[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: Parallel Build with GNU make

From: Martin Dorey
Subject: RE: Parallel Build with GNU make
Date: Tue, 2 Feb 2016 20:14:04 +0000

I'm not sure that I'll be adding anything that isn't a statement of the obvious 
but I can explain how we solve this in our environment.  We have the subfolders 
specified in a variable, SUBDIRS.  Another variable, 
BUILD_RECURSION_PREREQUISITES is synthesized from SUBDIRS to contain the like 
of recurse.%.subfolder1 recurse.%.subfolder2 etc.  BUILD_TARGET_STEMS contains 
phony target names that a user might pass to make, like "release" and "debug".  
These stems eventually form the % in those prerequisites, with something like:


Then I serialize the recursion, where appropriate, with this gruesome fragment:

JOIN_ADJACENT_PAIRS = $(filter-out @@:% %:@@,$(join $(addsuffix :,$(1) @@),@@ 
%,$(STEM),$(call JOIN_ADJACENT_PAIRS,$(2))))

-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Roland Schwingel
Sent: Monday, February 01, 2016 23:40
To: Eli Zaretskii; address@hidden
Subject: Re: Parallel Build with GNU make

Sorry for following up on my own post,but there has been a "bug" in the 
last sentence... See for correction below

Am 02.02.2016 um 08:35 schrieb Roland Schwingel:
> Hi...
> Eli Zaretskii wrote on 01.02.2016 20:14:17:
>  > Add "--debug=j" to the make command-line switches and see if it
>  > launches more than one command at a time.
>  >
>  > Anyway, your makefile.mak file can be written in such a way as to
>  > effectively prohibit parallelism, due to how it lays out dependencies.
> This is something I am struggeling on. I tried this a couple of times
> but could not make it working in a way that is satisfying for me, as I
> could not find the correct way to control parallelism. I am aware of
> .NOTPARALLEL but this might not be enough for me.
> I have this examplarily layout:
> subfolder1 - subfolder4 - sub4_1.c
>               sub1_1.c     sub4_2.c
>               sub1_2.c
>               ...
> subfolder2 - sub2_1.c
>               sub2_2.c
> subfolder3 - sub3_1.c
>               sub3_2.c
> main1.c
> main2.c
> main3.c
> All subfolders shall be visited recursively first. No parallelism is
> allowed in visiting folders. But when in a folder and it comes to
> concrete compiling parallelism should be allowed.
> This should result in this compileorder for the above example
> Visit subfolder1 (not in parallel)
> Visit subfolder4 (not in parallel)
> Compile sub4_1.c and sub4_2.c in parallel
> Return to subfolder1
> Compile sub1_2.c and sub1_2.c in parallel
> Return to main
> Visit subfolder2 (not in parallel)
> Compile sub2_1.c and sub2_2.c in parallel
> Return to main
> Vist subfolder3 (not in parallel)
> Compile sub3_1.c and sub3_2.c in parallel
> Return to main
> Compile main1.c,main2.c and main3.c in parallel
> When doing a make -j 4 for instance make is visiting all subfolders in
> parallel which is not desired. When I am using .NOTPARALLEL the whole
> subfolder is not compiled in parallel which is also not desired. Each
> folder visit is a concrete submake in my case. Each folder has its own
> makefile
> Any clue on how to achieve the flow I want to have? Visiting folders non
> recursive while compiling itself inside of the folders is recursive?
Should read:
Visiting folders non PARALLEL while compiling itself inside of the 
folders is IN PARALLEL.

Sorry and thanks again for your help,


Bug-make mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]