[Axiom-developer] aldor patches

[daly]

Pete,

I read "Recursive Make Considered Harmful" and he makes some good points.

In Axiom's case I'm not sure that a single, mother-of-all-files, Makefile
is a workable solution. 

First, he claims that a single Makefile is more efficient. But the Makefile
lookup/DAG build is swamped by the size of the Axiom build in the "clean"
case and a "make" of Axiom with a single file changed is very fast.

Second, he claims that the DAG edges get missed when recursive makes
are used. That is probably true in C using include files. He proposes
some solutions (VPATH, GCC -MM) that are not useful for lisp. Axiom
does have some "include"-like issues because lisp macros have to be
in the compile environment at the correct time. You'll notice that Axiom
builds "depsys". This is an image that includes the macros that are used.
Depsys is used to compile the Axiom lisp code where needed. Thus the
"include"-like issue is handled. Changing a depsys file will likely
trigger a complete system rebuild because it is very hard to compute
the macro vs file dependency tree. I certainly don't want that tree
expressed (and hand maintained) in the Makefile stanzas.

Similarly in the algebra subdirectory I originally started using the
Makefile stanzas to decide what needs to be recompiled. Given 1100
domains and their various dependencies this is impractical. In fact,
I would claim that hand-maintaining any dependency graph with 1100
nodes cannot work. And it clearly cannot scale. If Axiom grows by a
factor of 100 over the next 30 years that would mean 110000 stanzas
with who-knows-how-many dependencies.

Third, he misses a key point. While recursive makes are inefficient
they are conceptually manageable. Axiom is a huge system and some of
the Makefiles used to exceed 40000 lines (which brought Bill Page to
open rebellion and rightly so. I used a scheme which had a file-per-stanza
and his scheme uses generic rule patterns). I understand the details of
building Axiom and have expressed the process as clearly and correctly
as possible (modulo some complexity still to be handled like fixed-point
compiles). The directory structure and their associated Makefiles
reflect that understanding.

If I were to combine the Makefiles into one Mother-Makefile it would
probably total 100000 lines which would be a HUGE rule-based program
(maybe the largest I've ever seen and I authored a rule-based programming
language at IBM).

Recursion allows me to dole out the tasks in slightly more manageable
conceptual chunks. Each subdirectory only cares about itself and its
children so the frame of reference is localized.

I'll trade program inefficiency for clarity any day. As for "cross
directory dependency" I used "trigger files" to indicate that a
rebuild is needed. For instance, src/interp/Makefile has the 
'database.date' marker which is touched if the databases get
rebuilt and interpsys needs to be regenerated.

Miller assumes that you can code all of your dependency information
in the Makefile stanzas. In any really complex project this is not
possible (try hand-coding the axiom algebra lattice).

Miller rails against the practice of writing procedural control
related shell scripts in Makefiles which undermine the whole concept
of a rule-based programming language. On this point I agree wholeheartedly.

I've also tried hard to avoid all uses of special tools such as
#if, common includes, dynamic includes (well, 'findSpadFiles' in
src/algebra/Makefile.pamphlet is an exception), and other computed
approaches. Given the one-stanza-per-file method it turns out that
make has very little to do except stat files and build the well-specified
DAG.

In any case, I'm happy to see you didn't include a patch which rewrote
all of the Makefiles into one big Makefile. I'm old and my heart would
not stand such a shock.

t