Re: RFC: defining aclocal.m4/configure/config.status rebuild rules in all directories

[Alexandre Duret-Lutz]

>>> "adl" == Alexandre Duret-Lutz <address@hidden> writes:

 adl> The following patch applies on top of Akim's revised patch
 adl> (which I haven't installed yet).

 adl> It fixes the subdir8.test as it was introduced by the
 adl> aforementioned patch.  However I've updated subdir8.test to
 adl> show that the rebuild rules still aren't perfect.

Here is the revised version of this patch, that moves the
troublesome part of this test to aclocal5.test, so that it at
least shows that subdir8.test is solved.

[...]

 adl> One solution might be to encode aclocal.m4's dependencies in
 adl> aclocal.m4 itself.  aclocal would do this, and automake would
 adl> just have to grep aclocal.m4 for such dependencies.  
 adl> I'm not really fond of this solution because it seems
 adl> error prone, and aclocal is bound to disappear anyway.

It just occured to me there is a nifty way to fix this.  When
aclocal is about to copy a local file (where local means part of
the project), it can just replace it with
m4_include([the_local_file]).  Then, there is no reason to
rebuilt aclocal.m4 when these files changes: only configure need
to be rebuilt but as we keep track of m4_included file that sort
of things already work.  

Doing this would
 (1) simplify Automake: we no longer have to care about aclocal.m4's
     dependencies anymore, i.e., no need to parse ACLOCAL_AMFLAGS 
     and to glob *.m4 files in included directories (incidentally 
     this will fix our good old PR/11).
 (2) get closer to Akim's plan for getting rid of aclocal and replacing
     it with a bunch of m4_includes.
 (3) lessen the size of aclocal.m4 (not a big deal, but it's
     somewhat satisfactory to have only one copy of each macro
     in the package).

Index: ChangeLog
from  Alexandre Duret-Lutz  <address@hidden>

        For automake PR/46:
        * automake.in (scan_aclocal_m4): Work in all directories, not only
        the top-level.  Prepend $(top_srcdir) in front of each aclocal.m4
        dependency.
        * lib/am/configure.am (config.status, configure, $(ACLOCAL_M4)):
        Define rebuild rules in each directory.
        * tests/subdir8.test: Augment to exercise aclocal.m4's dependencies
        from the top-level directory.
        * tests/aclocal5.test: New file.  Exercise aclocal.m4's dependencies
        from a subdirectory.
        * tests/Makefile.am (TESTS): Add aclocal5.test.
        (XFAIL_TESTS): Replace subdir8.test by aclocal5.test.

Index: NEWS
===================================================================
RCS file: /home/adl/CVSROOT/LOCAL/HEAD3-20030413-1504/NEWS,v
retrieving revision 1.1
diff -u -r1.1 NEWS
--- NEWS        13 Apr 2003 13:04:19 -0000      1.1
+++ NEWS        13 Apr 2003 14:24:39 -0000
@@ -64,6 +64,24 @@
   substituted from configure.ac.  This has been requested by people
   dealing with non-POSIX ar implementations.
 
+* Any file which is m4_included from configure.ac will appear as
+  a configure and Makefile.in dependency.
+
+* The rules for rebuilding Makefiles and Makefile.ins will now rebuild
+  all Makefiles and all Makefile.ins at once when one of configure's
+  dependencies has changed.  This is considerably faster than
+  previous implementations, where config.status and automake were run
+  separately in each directory (this still happens when you change
+  a Makefile.am locally, without touching configure.ac or friends).
+  Doing this also solves a longstanding issue: these rebuild rule
+  failed to work when adding new directories to the tree, forcing
+  you to run automake manually.
+
+* For similar reasons, the rules to rebuild configure, config.status, and
+  aclocal.m4 are now defined in all directories.  Note that if you
+  were using the CONFIG_STATUS_DEPENDENCIES and CONFIGURE_DEPENDENCIES
+  (undocumented) variables, you will have to define them in all directories.
+  This is easily done using an AC_SUBST.
 
 New in 1.7:
 * Autoconf 2.54 is required.
Index: automake.in
===================================================================
RCS file: /home/adl/CVSROOT/LOCAL/HEAD3-20030413-1504/automake.in,v
retrieving revision 1.1
diff -u -r1.1 automake.in
--- automake.in 13 Apr 2003 13:04:19 -0000      1.1
+++ automake.in 13 Apr 2003 13:36:19 -0000
@@ -4282,22 +4282,13 @@
 {
     my $regen_aclocal = 0;
 
-    if (-f 'aclocal.m4')
-    {
-       # When using aclocal.m4, define this variable even in subdirectories,
-       # because every Makefile.in depends on $(ACLOCAL_M4).
-       &define_variable ("ACLOCAL_M4", '$(top_srcdir)/aclocal.m4', INTERNAL);
-    }
-
-    return (0, ())
-      unless $relative_dir eq '.';
-
     &examine_variable ('CONFIG_STATUS_DEPENDENCIES');
     &examine_variable ('CONFIGURE_DEPENDENCIES');
 
     if (-f 'aclocal.m4')
     {
        &push_dist_common ('aclocal.m4');
+       &define_variable ("ACLOCAL_M4", '$(top_srcdir)/aclocal.m4', INTERNAL);
 
        my $aclocal = new Automake::XFile "< aclocal.m4";
        my $line = $aclocal->getline;
@@ -4331,7 +4322,7 @@
                    foreach my $ac_dep (&my_glob ($amdir . '/*.m4'))
                    {
                        $ac_dep =~ s/^\.\/+//;
-                       push (@ac_deps, $ac_dep)
+                       push (@ac_deps, '$(top_srcdir)/' . $ac_dep)
                          unless $ac_dep eq "aclocal.m4"
                            || $ac_dep eq "acinclude.m4";
                    }
Index: lib/am/configure.am
===================================================================
RCS file: /home/adl/CVSROOT/LOCAL/HEAD3-20030413-1504/lib/am/configure.am,v
retrieving revision 1.1
diff -u -r1.1 configure.am
--- lib/am/configure.am 13 Apr 2003 13:04:20 -0000      1.1
+++ lib/am/configure.am 13 Apr 2003 14:05:07 -0000
@@ -66,37 +66,31 @@
 ## --------------------------- ##
 
 if %?TOPDIR_P%
-## Explicitly look in srcdir for benefit of non-GNU makes.
-## Use `$(top_builddir)' for the benefit of Tru64 v5.1 make and also
-## NetBSD v1.5 make.  These `make's don't know that
-## `$(top_builddir)/config.status' and `config.status' are the same
-## file when top_builddir==`.'.
-$(top_builddir)/config.status: $(srcdir)/configure 
$(CONFIG_STATUS_DEPENDENCIES)
-       $(SHELL) ./config.status --recheck
-
 ## Always require configure.ac and configure at top level, even if they
 ## don't exist.  This is especially important for configure, since it
 ## won't be created until autoconf is run -- which might be after
 ## automake is run.
 DIST_COMMON += configure %CONFIGURE-AC%
-
-## Explicitly look in srcdir for benefit of non-GNU makes.
-$(srcdir)/configure: %MAINTAINER-MODE% $(srcdir)/%CONFIGURE-AC% $(ACLOCAL_M4) 
$(CONFIGURE_DEPENDENCIES) %CONFIGURE_DEPS%
-       cd $(srcdir) && $(AUTOCONF)
 endif %?TOPDIR_P%
 
+$(top_builddir)/config.status: $(top_srcdir)/configure 
$(CONFIG_STATUS_DEPENDENCIES)
+       cd $(top_builddir) && $(SHELL) ./config.status --recheck
+
+$(top_srcdir)/configure: %MAINTAINER-MODE% $(top_srcdir)/%CONFIGURE-AC% 
$(ACLOCAL_M4) $(CONFIGURE_DEPENDENCIES) %CONFIGURE_DEPS%
+       cd $(top_srcdir) && $(AUTOCONF)
 
 
 ## ------------ ##
 ## aclocal.m4.  ##
 ## ------------ ##
 
-if %?TOPDIR_P%
-if  %?REGEN-ACLOCAL-M4%
-$(ACLOCAL_M4): %MAINTAINER-MODE% %CONFIGURE-AC% %ACLOCAL_M4_DEPS%
-       cd $(srcdir) && $(ACLOCAL) $(ACLOCAL_AMFLAGS)
-endif  %?REGEN-ACLOCAL-M4%
-endif %?TOPDIR_P%
+## aclocal.m4 must be built by the top-level Makefile, because this is
+## where the user is expected to define $(ACLOCAL_AMFLAGS).
+if %?REGEN-ACLOCAL-M4%
+$(ACLOCAL_M4): %MAINTAINER-MODE% $(top_srcdir)/%CONFIGURE-AC% %ACLOCAL_M4_DEPS%
+?TOPDIR_P?     $(ACLOCAL) $(ACLOCAL_AMFLAGS)
+?!TOPDIR_P?    cd $(top_srcdir) && $(MAKE) $(AM_MAKEFLAGS) '$$(ACLOCAL_M4)'
+endif %?REGEN-ACLOCAL-M4%
 
 
 ## --------- ##
Index: tests/Makefile.am
===================================================================
RCS file: /home/adl/CVSROOT/LOCAL/HEAD3-20030413-1504/tests/Makefile.am,v
retrieving revision 1.1
diff -u -r1.1 Makefile.am
--- tests/Makefile.am   13 Apr 2003 13:04:21 -0000      1.1
+++ tests/Makefile.am   17 Apr 2003 18:43:09 -0000
@@ -1,6 +1,6 @@
 ## Process this file with automake to create Makefile.in
 
-XFAIL_TESTS = subdir8.test auxdir2.test cond17.test
+XFAIL_TESTS = aclocal5.test auxdir2.test cond17.test
 
 TESTS =        \
 acinclude.test \
@@ -11,6 +11,7 @@
 aclocal2.test \
 aclocal3.test \
 aclocal4.test \
+aclocal5.test \
 acoutnoq.test \
 acoutpt.test \
 acoutpt2.test \
Index: tests/aclocal5.test
===================================================================
RCS file: tests/aclocal5.test
diff -N tests/aclocal5.test
--- /dev/null   1 Jan 1970 00:00:00 -0000
+++ tests/aclocal5.test 17 Apr 2003 18:42:50 -0000
@@ -0,0 +1,61 @@
+#! /bin/sh
+# Copyright (C) 2003  Free Software Foundation, Inc.
+#
+# This file is part of GNU Automake.
+#
+# GNU Automake is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# GNU Automake is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Automake; see the file COPYING.  If not, write to
+# the Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+# Boston, MA 02111-1307, USA.
+
+# Test to make sure that aclocal.m4's dependencies are honored
+# in sub-directories.
+
+# This test assumes that the `make' utility is able to start
+# over and reload Makefiles which have been remade (a non-POSIX feature).
+required='GNUmake'
+. ./defs || exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+MORE_DEFS
+AC_CONFIG_FILES([sub/Makefile])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+SUBDIRS = sub
+ACLOCAL_AMFLAGS = -I m4
+END
+
+mkdir sub
+: > sub/Makefile.am
+
+mkdir m4
+echo 'AC_DEFUN([MORE_DEFS], [])' > m4/moredefs.m4
+
+$ACLOCAL -I m4
+$AUTOCONF
+$AUTOMAKE --copy --add-missing
+./configure
+$MAKE
+
+# Update an aclocal.m4 dependency, then make sure all Makefiles
+# are updated, even from a sub-directory.
+echo 'AC_DEFUN([MORE_DEFS], [AC_SUBST([GREPME])])' > m4/moredefs.m4
+cd src
+$MAKE
+cd ..
+grep GREPME Makefile
+grep GREPME sub/Makefile
Index: tests/subdir8.test
===================================================================
RCS file: /home/adl/CVSROOT/LOCAL/HEAD3-20030413-1504/tests/subdir8.test,v
retrieving revision 1.1
diff -u -r1.1 subdir8.test
--- tests/subdir8.test  13 Apr 2003 13:04:31 -0000      1.1
+++ tests/subdir8.test  17 Apr 2003 18:35:44 -0000
@@ -36,10 +36,14 @@
 AM_PROG_CC_C_O
 AC_PROG_CC
 m4_include([confiles.m4])
+MORE_DEFS
 AC_OUTPUT
 END
 
-echo 'SUBDIRS = sub' > Makefile.am
+cat > Makefile.am << 'END'
+SUBDIRS = sub
+ACLOCAL_AMFLAGS = -I m4
+END
 
 mkdir sub
 
@@ -57,7 +61,10 @@
 }
 END
 
-$ACLOCAL
+mkdir m4
+echo 'AC_DEFUN([MORE_DEFS], [])' > m4/moredefs.m4
+
+$ACLOCAL -I m4
 $AUTOCONF
 $AUTOMAKE --copy --add-missing
 ./configure
@@ -89,3 +96,12 @@
 cd ..
 test -f maude/Makefile
 test -f sub/maude/Makefile
+
+# Make sure the dependencies of aclocal.m4 or honored at least from
+# the top-level directory.
+echo 'AC_DEFUN([MORE_DEFS], [AC_SUBST([GREPME])])' > m4/moredefs.m4
+$MAKE
+grep GREPME Makefile
+grep GREPME maude/Makefile
+grep GREPME sub/Makefile
+grep GREPME sub/maude/Makefile

-- 
Alexandre Duret-Lutz