Re: allowing users to add source files without rerunning the autotools?

[Daily, Jeff A]

On 1/12/12 2:16 PM, "Bill Sacks" <address@hidden> wrote:

> I work on a climate model (CESM); this is mostly written in Fortran 90, with a
> bit of C, but we are starting to add some C++ code, too. Until now, we have
> used our own, custom build system, but as the build becomes more complex, we
> are thinking of migrating towards greater use of the autotools.

Note that automake does not (yet?) support Fortran 90 dependency tracking.
I have raised this issue previously but those of us interested in developing
it are lacking time to work on it.  I am a fan of the autotools, but be
advised that CMake has its own Fortran parser for generating dependencies.
Now, if dependency tracking is not desired then this is no longer an issue.

> However, one challenge we face is the blurred line between developers and
> users. Many users need to modify the source code before building the model,
> and this can include adding entirely new source files. Existing code can
> depend on the new, user-created source files, and vice versa.
> 
> We implement this ability in our current build system (which does not use any
> of the autotools) through the use of two scripts, both of which are executed
> according to rules in the makefile: The first script builds a list of all
> source files residing in a set of pre-defined directories, and the second
> script parses these source files to generate dependency lists. This system
> isn't bulletproof, but it works well for us. In particular, note that this
> means that we do not need to explicitly list the source or object files in our
> makefile, since these lists are generated at build time.
> 
> My (admittedly limited) understanding of automake is that it requires all
> source files to be listed explicitly in Makefile.am, and thus any addition to
> this list would require rerunning automake, etc.

Yes, all sources must be listed, AFAIK.  You could write your own build
rules and targets for these files, which would be copied by automake into
your eventual Makefile, but that somewhat defeats the purpose of using a
build tool if you're writing many of the rules yourself.  This issue would
be similar with other build tools, having to rerun the tools if the lists of
source files change.

You could potentially generate a makefile fragment containing your list of
sources and "include" it from your automake-generated Makefile, but you'd
need a hacky work-around since "include" is also an automake directive.  See
the following discussion:
http://old.nabble.com/ifdef-...-endif-issue-td12330117.html

> We are wondering if it will be possible for us to support the functionality of
> user-added source files in an autotools-based build, without requiring users
> to rerun the full autotools chain. Is this possible? We can assume that
> user-created source files do not change any of the options required for the
> build (e.g., there would be no new library dependencies).

What if you built a separate library representing the user code?  The
interdependencies between user code and original code would be tricky.
There's nothing wrong with having automake build the rest of the code and
have a recursive make call to a hand-rolled Makefile.

> I have read the FAQ entry on why automake doesn't support wildcards, so I'm
> wondering if there is some other workaround to achieve the functionality we
> want.
> 
> Thank you,
> Bill

I tried not to ramble here but rather offer hopefully a few useful ideas...