help-make
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

lRE: GNU Make 4.2 Query


From: Cook, Malcolm
Subject: lRE: GNU Make 4.2 Query
Date: Sat, 23 Jan 2021 16:35:45 +0000

Thanks, Paul, for the open-ended request for suggestions.... 

I am most interested in improved control over parallelization.    I expect 
others have suggested things that have your consideration already, am I right, 
and are any of the proposals "worthy" in your estimation?  

Perhaps the ability to declare on a {static,pattern} rule level, the load 
number of CPUs the rule body is intended to us, allowing Make to use that 
number as a multiplier in determining whether running the job will 
(theoretically) saturate --jobs.  Perhaps a convention that the variable 

--jobs and .NOTPARALLEL are the only control I see, and it is very gross level 
of control.  I have many cases where different "steps" of a workflow can be 
parallelized internally, and have few options to communicate that to Make.  

FWIW, when I have a pattern rule which is highly parallelizable, I have figured 
out how to force all matching targets to run in serial by making them depend 
upon one another in serial with a custom macro, .  Kind of a hack, but it 
allows me to have only one instance of the recipe running at a time by writing:

$(call NotParallel,${matchingTargets})

What might be useful would be to have a variable MAKEJOBSLOTS which defaults to 
1 and declares the number of slots/cpus each job is expected/designed to 
consume, and then allow that variable to appear as pattern rules specific 
variable.

Thoughts?

On the topic, do you want to warn me against doing things like this?:

.j=$(subst ${space},,$(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS))))
### Exposes the value of -j to the makefile (e.g. to reason about
### process allocation).  NB: will be empty if not passed - does not
### default.

Finally, if you or anyone is interested, or want to give me suggestions for 
improvement, NotParallel is defined as:

################################################################################
## ALAS: .NOTPARALLEL: is global.  Here is a hack to allow only
## selected rules to be declared as NOTPARALLEL.
##
## First, we need a macro to ensure that ${1} is created AFTER ${2} is
## completed:
NotParallelDepends=${1}: | ${2}
## Then we need a macro to call NotParallelDepends on pairs of successive
## arguments:
NotParallel=$(foreach i,$(shell seq 2 $(words ${1})),$(eval $(call 
NotParallelDepends,$(word ${i},${1}),$(word $(shell echo $$(( ${i} - 1 ))) 
,${1}))))

Cheers, and good luck with the next rev!

Malcolm Cook

reply via email to

[Prev in Thread] Current Thread [Next in Thread]