Re: Recursive Make Considered Harmful [Long Post]

[gk]

At 11:29 AM 10/22/2002 +0200, Johan Bezem wrote:
> Section 2 lists the problems PM sees:
> - The first one is a problem itself, but you can specify dependencies
> between _directories_ (not files!) in GNUMake like this:
>
>   module1/ : module2/
>
> So, if you need 'module2' to be built before 'module1', say so. I did
> this and tested on various platforms. Mind the slashes.

I think we are all trying to just get a job done and whatever works is 
often all we want at the moment.
But this kind of manual tweaking is error-prone: somebody has to:
1. Discover the problem by testing or being bitten by a bug
2. Remember to make the tweak, or hope that somebody else gets around to it.

I like the single makefile approach better because you NEVER have an issue 
of module order to maintain.

> If a close interaction between two modules as in the sample in
> 3.2 is necessary, the two modules belong in one: The decomposition
> is wrong.

I think that is precisely Miller's argument: don't ever decompose and you 
don't have to worry.
I think Miller considered the idea of fixing the decomposition problem by 
going half way, and fixing it - but why bother with half-way approaches 
when you can simply extend the concept all the way and be done.

His example may be 'wrong' decomposition, but it was meant to illustrate 
the point about how a system can evolve over time:
* initially the decomposition was fine, but it is possible that a 
prerequisite with NO PREREQUISITES might come to have prerequisites.

Suppose a prerequisite, myFile, with no prerequisites, is shared through a 
system, in different makefiles.
make doesn't keep a database of all targets that depend on that 
prerequisite so when somebody comes along and decides to generate myFile 
somehow:
myFile : anotherFile

At this point somebody must REMEMBER to search all the makefiles to find 
the places where myFile is a prerequisite and then tweak the recursive 
build order or add rules to those makefiles. What makes matters worse is 
that a simple text search won't reliably find the answer since the same 
name 'myFile' may refer to different files in different makefile contexts.

The single makefile approach neatly solves the uncertainties - harder to 
setup initially than recursion but far more scaleable.
As an added benefit, since the single makefile approach uses unique 
pathnames for all files, you actually could do a text search and find all 
dependent files if you wanted to, but you don't need to anymore.

I think the key for the single makefile concept to reduce maintenance is to 
have all the makefiles auto-generated as things change.

I am working toward a single makefile system that I hope will scale well. 
Here is a simplified outline.

* I wrap Gnu make in a shell script, 'xmlmake', that finds a configuration 
file in the current path and sets up the environment. In particular:
export PROJECT_DIR=/top # or whatever directory where the project config 
file was found
export PROJECT_MAKEFILE=/top/xmProject.mk # I don't use Gnu makefile names 
to allow Gnu make systems to share the same space.
The command:
$xmlmake <options> <targets>
Results in calling Gnu make as:
$/usr/bin/make -f "$PROJECT_MAKEFILE" <options> <targets>

* $PROJECT_MAKEFILE includes all the other makefiles in the system.

* all filenames in explicit rules use an absolute path prefixed by 
$(PROJECT_DIR):
E.g., $(PROJECT_DIR)/path/to/file

* Every source file has one dependency makefile which describes the 
prerequisites of its output file(s).
If implicit rules can build the output(s) without a dependency makefile, 
one might not be created since including a non-existent makefile is not a 
problem.

* Each source file type defines unique suffixes for source and dependency 
makefiles.

* The top level makefile knows how to auto-generate makefiles when new 
sources appear.

Suppose the output 'a.xmo' also depends on a file in another directory, 
'myFile'.
Initially 'myFile' has no prerequisites.
Dependency makefile, 'a.xms.d' is automatically generated from a source 
file 'a.xms'.
File 'a.xms.d' contains:
$(PROJECT_DIR)/path/to/a.xmo : \
$(PROJECT_DIR)/path/to/a.xms \
$(PROJECT_DIR)/path/to/a.xms.d \
$(PROJECT_DIR)/another/path/to/myFile

Suppose many other files depend on $(PROJECT_DIR)/another/path/to/myFile.
Now suppose somebody decides to generate myFile from myFile.mys.

No need to update a.xms.d at all. (or any other dependency makefile.)
No need to tweak the build order since all makefiles are always included.

The top level makefile needs to be given a couple of things:
* method of learning that myFile.s exists and therefore 'myFile.mys.d' 
needs to be auto-generated:
        scan project directories for new files only when the directories 
change
* pattern rule for building dependency files: '%.mys.d' from '%.mys'
* rule for compiling source files '%.mys'

* save the list of directories in an included makefile, for efficiency:
PROJECT_DIRS:= \
$(PROJECT_DIR) \
$(PROJECT_DIR) /subdir1 \
$(PROJECT_DIR) /subdir1/a \
$(PROJECT_DIR) /subdir1/b \
$(PROJECT_DIR) /subdir2 \
...
.PHONY: subdirs
subdirs : $(PROJECT_DIRS)
        # re-create this makefile with current list of project directories
        # filter out any directories like CVS, etc.

I see no value for sub-make except in the case where a system already uses 
it and you don't want to bother changing things because they aren't broken 
(yet).
- Greg Keraunen <xmlmake.com>