[Axiom-developer] Re: aldor patches

[Peter Broadbery]

No problem - I've a project which is suitable for non-recursive
makefiles, and nicked the structure from there.  As you say, axiom in
its entirety is not really the kind of thing that can be tackled this
way.  The reference to 'considered harmful' was just to a similar style,
not signalling any intent to change existing code, and I've no argument
with your assesment below.

In the case of the aldor library the non-recursive part is less
important than the ability to list a bunch of files and have make sort
out what targets you mean;  this way I don't need to autogenerate
stanzas, and the build will include new types as they are added to
src/algebra.  Both of these should reduce the amount of maintenance
required by libaxiom, which was an important goal.

Peter

On Mon, 2005-08-22 at 23:01 -0500, address@hidden wrote:
> Pete,
> 
> I read "Recursive Make Considered Harmful" and he makes some good points.
> 
> In Axiom's case I'm not sure that a single, mother-of-all-files, Makefile
> is a workable solution. 
> 
> First, he claims that a single Makefile is more efficient. But the Makefile
> lookup/DAG build is swamped by the size of the Axiom build in the "clean"
> case and a "make" of Axiom with a single file changed is very fast.
> 
> Second, he claims that the DAG edges get missed when recursive makes
> are used. That is probably true in C using include files. He proposes
> some solutions (VPATH, GCC -MM) that are not useful for lisp. Axiom
> does have some "include"-like issues because lisp macros have to be
> in the compile environment at the correct time. You'll notice that Axiom
> builds "depsys". This is an image that includes the macros that are used.
> Depsys is used to compile the Axiom lisp code where needed. Thus the
> "include"-like issue is handled. Changing a depsys file will likely
> trigger a complete system rebuild because it is very hard to compute
> the macro vs file dependency tree. I certainly don't want that tree
> expressed (and hand maintained) in the Makefile stanzas.
> 
> Similarly in the algebra subdirectory I originally started using the
> Makefile stanzas to decide what needs to be recompiled. Given 1100
> domains and their various dependencies this is impractical. In fact,
> I would claim that hand-maintaining any dependency graph with 1100
> nodes cannot work. And it clearly cannot scale. If Axiom grows by a
> factor of 100 over the next 30 years that would mean 110000 stanzas
> with who-knows-how-many dependencies.
> 
> Third, he misses a key point. While recursive makes are inefficient
> they are conceptually manageable. Axiom is a huge system and some of
> the Makefiles used to exceed 40000 lines (which brought Bill Page to
> open rebellion and rightly so. I used a scheme which had a file-per-stanza
> and his scheme uses generic rule patterns). I understand the details of
> building Axiom and have expressed the process as clearly and correctly
> as possible (modulo some complexity still to be handled like fixed-point
> compiles). The directory structure and their associated Makefiles
> reflect that understanding.
> 
> If I were to combine the Makefiles into one Mother-Makefile it would
> probably total 100000 lines which would be a HUGE rule-based program
> (maybe the largest I've ever seen and I authored a rule-based programming
> language at IBM).
> 
> Recursion allows me to dole out the tasks in slightly more manageable
> conceptual chunks. Each subdirectory only cares about itself and its
> children so the frame of reference is localized.
> 
> I'll trade program inefficiency for clarity any day. As for "cross
> directory dependency" I used "trigger files" to indicate that a
> rebuild is needed. For instance, src/interp/Makefile has the 
> 'database.date' marker which is touched if the databases get
> rebuilt and interpsys needs to be regenerated.
> 
> Miller assumes that you can code all of your dependency information
> in the Makefile stanzas. In any really complex project this is not
> possible (try hand-coding the axiom algebra lattice).
> 
> Miller rails against the practice of writing procedural control
> related shell scripts in Makefiles which undermine the whole concept
> of a rule-based programming language. On this point I agree wholeheartedly.
> 
> I've also tried hard to avoid all uses of special tools such as
> #if, common includes, dynamic includes (well, 'findSpadFiles' in
> src/algebra/Makefile.pamphlet is an exception), and other computed
> approaches. Given the one-stanza-per-file method it turns out that
> make has very little to do except stat files and build the well-specified
> DAG.
> 
> In any case, I'm happy to see you didn't include a patch which rewrote
> all of the Makefiles into one big Makefile. I'm old and my heart would
> not stand such a shock.
> 
> t
-- 
Peter Broadbery <address@hidden>