Re: Why no dependency tracking with preprocessed Fortran?

[Ralf Wildenhues]

* Daily, Jeff A wrote on Thu, Jun 24, 2010 at 04:46:28PM CEST:
> > > Why doesn't automake have dependency tracking for preprocessed
> > > Fortran?
> > 
> > Because nobody has implemented it yet.

> Ok, so how does one go about developing this sort of thing for
> Automake?  I'm certainly interested -- having time to do the work is
> always the issue, right?

Actually, for Automake, having a good idea about the problem space is
often the critical part of the job.  If I don't have that, I tend to be
overly sceptic of any patches, OTOH if I think we have a good strategy,
then doing the actual implementation is often rather straight-forward.

> I've found a number of tools which claim to do dependency tracking:
> 
> Foray http://code.google.com/p/foraytool/
> Makedepf90 http://personal.inet.fi/private/erikedelmann/makedepf90
> TCBuild http://macresearch.org/tcbuild-new-build-tool-fortran (precursor to 
> foray?)
> mkmf http://www.gfdl.noaa.gov/~vb/mkmf.html
> fmkmf http://www.geos.ed.ac.uk/homes/hcp/fmkmf/
> 
> But what would be the best strategy for Automake?  I've seen
> discussion of this earlier:
> 
> http://www.mail-archive.com/address@hidden/msg00682.html
> 
> In short, I'm asking for advice.  If I want the best chances of
> getting this feature (include as well as module dependency tracking in
> Fortran 77/90) into Automake, what's the suggested way to implement
> it?

Step 0: read the Automake manual History chapter about Dependency
Tracking Evolution.  In Detail.  We don't want to make a mistake
again that was already made with C.

For preprocessed Fortran, the remaining bits are the system- and
compiler-specific tricks to get the header dependency information.
Enumerate them, how they work, quirks, whether and which compilers
can be made to produce dependency information as side effects.
The rest should be straight-forward.

Conditional Compilation[2] is standardized but not widely used; the
coco tool[3] may be used for this.

Fortran modules are another matter.  From what I have understood so far:

Fortran files may define modules and use other modules.  In the most
general case, one Fortran source file can define an arbitrary set of
modules, and use an arbitrary set of modules, there is no required
naming strategy.  If you draw arrows between .f90 files from 'use'
statements to 'module' statements, this graph (which is not the one
'make' is interested in!) can have circles.  Is this true?  If yes,
then I hope this is frowned upon in the Fortran community.

Fortran compilers typically encode module information in some manner.
It may be in extra module files, which may have one of several possible
extensions (I only know .mod) and may be named after the module name
(also the object name possibly?  I don't know).  Alternatively, module
information might be encoded inside object files (Cray compiler[1]), or
kept inside memory when doing whole-program compilation.

Fortran compilers that write external modules typically allow to add to
a search path to find such files (-I or so) and allow to specify a
directory to write module files to (-J/-M (gfortran), -qmoddir (xlf),
-module (ifort)).

There are probably more semantics, but I'm not aware of them.

Some compilers may be (or become[1]) smart enough to produce such
dependency information, otherwise an external tool or a small script
could do that.

Fortran module files typically have to be cleaned.  Since they are
compiler-dependent, and effectively binary like object files, they are
typically not distributed.

I have no idea how modules mesh with static or shared libraries.  I have
seen module files installed, typically below $(includedir), but that
seems not very well thought-through, because I've had trouble avoiding
them when using a different compiler from the one used to generate the
modules.  Oh well, that may be a limitation users have to live with
anyway.

For 'make' purposes, the dependency situation could mean that we need a
step before normal compilation, or that we could limit allowed semantics
for Automake projects and go about in some predefined strategy.  I think
it's too early to decide here, before we know better about above.

For example, if we could rely on module information being encoded in
external files, say foo.mod ones, then there are dependencies of bar.o
on foo.mod if bar.f90 has 'use foo'.  Even with circular graphs above,
there is a way out here by pre-creating some of the .mod files.  In
fact, we may introduce a mode that plays safe and precreates dependency
information if need be, and a mode which doesn't need this.

Generally, I would prefer to avoid getting in the situation where we
parse Fortran sources ourselves unless we have to.  Always prefer side
effect work, and prefer better tools, esp. compilers, to help us.  If
there are no better tools, then let's write a shell script helper to do
the job for us.

If possible, 'automake' should avoid having to look into source file
contents as much as possible (effectively making Makefile.in depend upon
source files); a tool that runs at 'make' time should do that.  Any tool
that runs at 'make' run time should use portable tools only (info
standards "Utilities in Makefiles").

We may have to impose some restrictions at least on recursive makefile
setups.  That is fine with me, as long as usual convention setups are
possible.

You should sign copyright papers if you really want to work on this
(more on this off-list).  If you do, read the HACKING file in the git
tree.  Generally, all new features and code paths should be exercised by
the test suite, have appropriate documentation etc.  In fact, it's good
style to write tests first, then adjust code to work with them.

Cheers,
Ralf

[1] http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42607
    http://gcc.gnu.org/bugzilla/show_bug.cgi?id=43954
[2] info gfortran "Preprocessing and conditional compilation"
[3] http://users.erols.com/dnagle/coco.html