[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: 4.4.1 breaks recursive invocation with --print-directory [when addin
From: |
Paul Smith |
Subject: |
Re: 4.4.1 breaks recursive invocation with --print-directory [when adding to MAKEFLAGS] |
Date: |
Wed, 01 Mar 2023 17:28:31 -0500 |
User-agent: |
Evolution 3.46.4 (by Flathub.org) |
On Wed, 2023-03-01 at 13:05 -0600, Satish Balay wrote:
> Perhaps my minimal test code is not an exact representation. However
> this usage does work in our code-base.
>
> > > >
> [balay@pj01 petsc]$ make --version |head -1
> GNU Make 4.3
> [balay@pj01 petsc]$ grep ^MAKE_NP configure.log
> MAKE_NP = 6
> [balay@pj01 petsc]$ (ccache -c -C && make clean) > /dev/null && time
> (make all) > /dev/null
>
> real 1m15.591s
> user 6m4.442s
> sys 1m9.179s
> [balay@pj01 petsc]$ (ccache -c -C && make clean) > /dev/null && time
> (make -j1 all) > /dev/null
>
> real 6m5.378s
> user 5m5.263s
> sys 0m59.175s
> [balay@pj01 petsc]$ (ccache -c -C && make clean) > /dev/null && time
> (make -j12 all) > /dev/null
>
> real 1m8.758s
> user 10m58.865s
> sys 1m41.094s
> <<<
>
> Note: '-j6' is the value set in the sub-makefile for all the above
> runs.
All of this behavior makes sense and aligns with what I wrote in my
previous email message, with one caveat I didn't think of.
What happens is this:
In the first instance where you don't provide -j at all, the sub-make
(that adds -j6) will create a new jobserver with 6 tokens and that make
and all its sub-makes will share that jobserver.
In the second instance where you provide -j1, the arguments passed to
the sub-make (after you've modified them) will be: "-j6 -j1" because "-
j1" appears in makeflags. I didn't consider this idea of explicitly
passing "-j1" on the command line.
In the third instance where you provide -j12, a new jobserver will be
created with 12 tokens and the top-level make and all its sub-makes
will share that jobserver, regardless of the value of -j added by the
sub-make.
So, the only time that the MAKEFLAGS="-j6 $$MAKEFLAGS" has a different
behavior than MAKEFLAGS="$$MAKEFLAGS -j6" is in the case where someone
invokes "make -j1" at the top level; in the first case this will
prevent the sub-make from starting a jobserver, while in the second
case it will still do that. Otherwise it has no effect.
You can prove this to yourself with this simple makefile:
$ cat Makefile
.RECIPEPREFIX := >
top:
> @echo "$@: $$MAKEFLAGS"
> $(MAKE) middle
middle:
> @echo "$@: $$MAKEFLAGS"
> MAKEFLAGS="-j4 $$MAKEFLAGS" $(MAKE) bottom
bottom: 1 2 3 4
> @echo "$@: $$MAKEFLAGS"
1 2 3 4:
> @echo $@ start; sleep 2; echo $@ end
First we run make with no -j at all (this is GNU make 4.3 FYI):
$ make --no-print-directory
top: --no-print-directory
make middle
middle: --no-print-directory
MAKEFLAGS="-j4 $MAKEFLAGS" make bottom
1 start
2 start
3 start
4 start
1 end
2 end
3 end
4 end
bottom: -j4 --jobserver-auth=3,4 --no-print-directory
You can see from the output that no jobserver was available in the
"middle" invocation, and that the "bottom" invocation was run with a
jobserver with 4 slots available (all four jobs started in parallel).
Now if we run with -j1:
$ make --no-print-directory -j1
top: -j1 --no-print-directory
make middle
middle: -j1 --no-print-directory
MAKEFLAGS="-j4 $MAKEFLAGS" make bottom
1 start
1 end
2 start
2 end
3 start
3 end
4 start
4 end
bottom: -j1 --no-print-directory
Now, the invocation of the jobserver in "middle" was ignored because
the -j1 option on the command line overrode the -j4 in the "middle"
rule. And we can see the "bottom" targets were all run serially, not
in parallel.
Finally, add -j2 to the parent:
$ make --no-print-directory -j2
top: -j2 --jobserver-auth=3,4 --no-print-directory
make middle
middle: -j2 --jobserver-auth=3,4 --no-print-directory
MAKEFLAGS="-j4 $MAKEFLAGS" make bottom
1 start
2 start
1 end
2 end
3 start
4 start
3 end
4 end
bottom: -j2 --jobserver-auth=3,4 --no-print-directory
Here you can see that the "-j4" option in "middle" was ignored. But if
you reverse the MAKEFLAGS assignment:
> MAKEFLAGS="$$MAKEFLAGS -j4" $(MAKE) -f /tmp/x8.mk bottom
You'll get the same results:
1 start
2 start
1 end
3 start
2 end
4 start
3 end
4 end
It still only runs with -j2 even though -j4 "should" take precedence.