[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Parallel Build with GNU make

From: Howard Chu
Subject: Re: Parallel Build with GNU make
Date: Tue, 2 Feb 2016 20:34:53 +0000
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Firefox/45.0 SeaMonkey/2.42a1

Martin Dorey wrote:
-----Original Message-----
From: address@hidden [mailto:address@hidden On Behalf Of Roland Schwingel
Sent: Monday, February 01, 2016 23:40
To: Eli Zaretskii; address@hidden
Subject: Re: Parallel Build with GNU make

Sorry for following up on my own post,but there has been a "bug" in the
last sentence... See for correction below

Am 02.02.2016 um 08:35 schrieb Roland Schwingel:

Eli Zaretskii wrote on 01.02.2016 20:14:17:

  > Add "--debug=j" to the make command-line switches and see if it
  > launches more than one command at a time.
  > Anyway, your makefile.mak file can be written in such a way as to
  > effectively prohibit parallelism, due to how it lays out dependencies.
This is something I am struggeling on. I tried this a couple of times
but could not make it working in a way that is satisfying for me, as I
could not find the correct way to control parallelism. I am aware of
.NOTPARALLEL but this might not be enough for me.

I have this examplarily layout:

subfolder1 - subfolder4 - sub4_1.c
               sub1_1.c     sub4_2.c
subfolder2 - sub2_1.c
subfolder3 - sub3_1.c

All subfolders shall be visited recursively first. No parallelism is
allowed in visiting folders. But when in a folder and it comes to
concrete compiling parallelism should be allowed.

This should result in this compileorder for the above example
Visit subfolder1 (not in parallel)
Visit subfolder4 (not in parallel)
Compile sub4_1.c and sub4_2.c in parallel
Return to subfolder1
Compile sub1_2.c and sub1_2.c in parallel
Return to main
Visit subfolder2 (not in parallel)
Compile sub2_1.c and sub2_2.c in parallel
Return to main
Vist subfolder3 (not in parallel)
Compile sub3_1.c and sub3_2.c in parallel
Return to main
Compile main1.c,main2.c and main3.c in parallel

When doing a make -j 4 for instance make is visiting all subfolders in
parallel which is not desired. When I am using .NOTPARALLEL the whole
subfolder is not compiled in parallel which is also not desired. Each
folder visit is a concrete submake in my case. Each folder has its own

Any clue on how to achieve the flow I want to have? Visiting folders non
recursive while compiling itself inside of the folders is recursive?
Should read:
Visiting folders non PARALLEL while compiling itself inside of the
folders is IN PARALLEL.

Use a for-loop for anything that must be explicitly serial.

Generally, to me this is a sign of a broken Makefiles - when you define your dependencies correctly, everything should naturally run in the correct order, regardless of degree of parallelism. The fact that you want subfolder 2 to be built after subfolder 1 means sub1 *is* a dependency of sub2 and if you haven't stated that in your Makefile then you've made an error in your Makefile.

Read this:
in particular this

  -- Howard Chu
  CTO, Symas Corp.           http://www.symas.com
  Director, Highland Sun     http://highlandsun.com/hyc/
  Chief Architect, OpenLDAP  http://www.openldap.org/project/

reply via email to

[Prev in Thread] Current Thread [Next in Thread]