Re: non recursive makefiles

[Nicola Pero]

> > Thanks, MJ Ray -- your comments make a good point -- the main problem is
> > that 'make' does not provide you with a full-fledged programming language.
> 
> Recursion or not doesn't change that either way.  If you have a specific
> point about a frequently-necessary feature that requires recursion,
> please restate it.

Well, I tried to explain in words.  You can't iterate in make.  You can't
execute pieces of code iteratively -- you can only execute them once, or
never.  this does not make it exactly easy to iterate over files, read
each of them, and then execute the same rules for each of them with
different variables.  You CAN NOT execute the same rules with different
variables in the same make invocation, which is what we want to do here.

So the only way of doing this is by recursive make invocations.

Moreover, there are no local variables or rules.  Everything is global.  
And variables in expressions are replaced at the point the expression is
used, not when is defined.  So if you use a variable with the same name
twice, even if think you no longer need the first one, you might actually
be messing things up because you are affecting every place where the first
one was used.  That poses so many additional problems if you don't have a
way of sandboxing each compilation run.  Implementation problems because
you have to try make at least internal variables used local to that run.  
So you prefix everything and everything gets so clumsy and you will still
get headaches.

Not only huge implementation problems inside gnustep-make, but imagine
from the user point of view - a mistyped variable in
Xxx/Yyy/Zzz/GNUmakefile could cause compilation in
Xxx/Kkk/Www/Hhh/GNUmakefile to fail with a mysterious error.

Sandboxing each make execution is obviously a lot better, it certainly
makes the code easier to understand and debug.  You have a point that
performance would be better if we could get away without it.

Btw, how much better ... are you sure that that is what gives you slow
build times ?  For me, GCC and autogsdoc are the performance killers
during compilation, make is fast enough even on a large tree.  I can't
turn off GCC :-) but I constantly turn off autogsdoc while developing, and
only turn it on again when deploying.

Also, if you are in desperate need of building performance and
gnustep-make is the culprit, strip the makefiles when installing (it's a
./configure option).  And more importantly make sure you don't have any
custom code which is spanning shell commands multiple times because of
incorrect usage of make variables ... (I'll help you privately off-list
with that if you need)

I'd stop the discussion here - if you are not convinced, look at the
source code.

 
> > It's extremely limited and we're already pushing it over the edge.  The
> > first obvious objection is that if you remove recursive invocations and
> > just include everything in the top-level GNUmakefile, then everything
> > defined in any GNUmakefile included by the top-level execution would be in
> > the same namespace.
> 
> Yes, the subprojects that you use in one project have to be compatible,
> but that's already true.

No, it's not.  You can do

ADDITIONAL_OBJCFLAGS += -lPincoPallino

in one GNUmakefile, and that will *not* be seen by another GNUmakefile in 
another directory.

If you included the two GNUmakefiles in the same make invocation, the 
variables would be shared, so an ADDITIONAL_OBJCFLAGS += in one 
GNUmakefile would affect the ADDITIONAL_OBJCFLAGS of the other one as 
well.

Mumble.  I wonder if I can explain myself clearly enough.


> Recursion doesn't change that fundamentally either.

Of course it does.  ADDITIONAL_OBJCFLAGS defined in subproject A have no
effect on ADDITIONAL_OBJCFLAGS defined in subproject B because they are
read in different, sandboxed, make invocations.


> A higher-level makefile can easily pollute the environment
> variables in a way that breaks lower levels, can't it?

Sure.  But there is a difference between "everything can influence/pollute
everything" (which means any GNUmakfile at any level can influence/pollute
any other GNUmakefile at any other level) and "everything can
influence/pollute only stuff at levels below" (which means you can have
the top-level GNUmakefile define top-level global flags and policies, but
subprojects are indipendent and don't mess up with each other)


> Not isolation at all. Assuming isolation seems dangerous to me.

I don't agree.  "Isolation" is a basic design pattern of good software
architecture, and it looks like an essential requirement for a maintable,
usable system to assume isolation of subproject makefile code.

Currently, "isolation" of submake invocations of different subprojects is
not just assumed, it's _guaranteed_.

Of course you can put top-level flags which are propagated to all
subinvocations in the top-level GNUmakefile, in the same way that you can
add a class method (or variable if you have that) to a class and that
applies to all instances; still, all instances are isolated between
themselves and that's good.

 
> Recursion does allow "dirty" makefile practices to continue longer. Maybe
> sometimes recursion would be necessary, but I don't see why it seems to
> be the default for so many GNUstep packages. Is it just lack of time to
> hack an elegant solution, which is what your message suggests?

No - it's the way that make works that forces you to do this.

make is supposed to let you define rules and targets and use them.  (btw,
generally, make does encourage you to use recursive invocations to iterate
over directories).

gnustep-make is trying to act as a sort of pre-processor: you define a few
variables, and gnustep-make will generate all the rules for you.  But
instead of gnustep-make being a preprocessor which generates a Makefile
from your Makefile.am (the way automake does it, for example),
gnustep-make will include the rules your makefile needs, and use your
variables to adapt them to your specific needs on the fly.  That way
gnustep-make is a lot faster than something like Automake, because it
skips the compilation step (ever tried using automake ? that really slows
your development cycle down) and you can execute your GNUmakefile directly
without compilation.

But there are limitations.  gnustep-make is basically including the code
fragments with the rules that your GNUmakefile needs.  But then the
customizations which are allowed go much further than that, and a lot of
those are represented as 'ifeq ($xxx, yyy) / dothis / else / dothat/ fi'
where dothis / dothat might be different rules or different replacements
or different inclusions that are used in the different cases, depending on
your variables.

That generates a lot of complexity, and to keep the complexity manageable,
you want to isolate different runs anyway.  The fact that make's syntax
and semantics are really designed for a fixed set of rules + some
variables and hacks, and do not really provide much high level support for
programming or "meta" code means what would be just a good design idea
becomes a necessity.


> The recursion in the current gnustep-make are part of the hindrance,
> but there's a lot more undocumented hacks and magics in there, which is
> probably the biggest reason it doesn't get all the love it could.

I don't understand your point here except for a generic complain that you
don't like recursive invocations and that the code is hard to understand
(I can tell you if it wasn't using recursive invocations it would be
incredibly more complex and difficult to understand).  And btw
gnustep-make gets a lot of love.  :-)

... maybe not recently as I didn't have much time.  I acknowledge that and
apologize.