bug#7884: [PATCH] yacc: fix VPATH builds with FreeBSD make (was: Re: bug#7884: Yacc and FreeBSD make in VPATH builds)

[Stefano Lattarini]

On Thursday 27 January 2011, Ralf Wildenhues wrote:
> Does 'make -n' create files (for any target)?
>
No, and I've updated the `yaccdry.test' testcase to also check for this
(see attached amended patch).

> How do you ensure in your patch that 'make distdir' does not create files?
>
Well, it does, but I don't see any problem with this.  It's perfectly normal
for 'make distdir' to create files in the build directory, no?  And anyway,
this behaviour was pre-existint with FreeBSD make, just in a much worser
form ('yacc' was used to recreate the files, instead of a simple "cp -p").

> How do you ensure in your patch that 'make clean' does not create files?
>
In fact, it does (good catch, BTW); see new testcase in the attached patch.
Yes, this is clearly a wart, but a minor one, and having it is IMHO a price
worth paying in order to have VPATH builds work on FreeBSD.

BTW, if you decide to get rid of said wart in a follow-up patch, that can
probably be done using the FreeBSD make special variable `.TARGETS' (akin
to GNU make's `MAKECMDGOALS').

Thanks,
  Stefano

From e5b393aaf4fe417e8cc8cae8056829f9083cc51c Mon Sep 17 00:00:00 2001
From: Stefano Lattarini <address@hidden>
Date: Thu, 27 Jan 2011 20:30:56 +0100
Subject: [PATCH] yacc: fix VPATH builds with FreeBSD make

Fixes automake bug#7884.

* lib/am/yacc.am (.BEGIN): New target, hack to support VPATH
builds with FreeBSD make.
* automake.in (lang_yacc_target_hook): Now update the private
make variable `am__yacc_c_outputs', used by the target above.
* tests/yacc-dist-nobuild.test: Update.
* tests/yaccdry.test: Extend.
* tests/yaccvpath.test: Likewise.
* tests/yacc-d-vpath.test: Likewise.
* tests/yacc-subdir-objects-vpath.test: New test.
* tests/yacc-cond.test: Likewise.
* tests/yacc-cond-minimal.test: New test, still failing with
BSD make.
* tests/yaccdry2.test: Likewise.
* tests/Makefile.am (TESTS): Update.
---
 ChangeLog                            |   19 +++++
 automake.in                          |    7 ++
 lib/am/yacc.am                       |   20 +++++
 tests/Makefile.am                    |    4 +
 tests/Makefile.in                    |    4 +
 tests/yacc-cond-minimal.test         |   81 ++++++++++++++++++++
 tests/yacc-cond.test                 |  122 ++++++++++++++++++++++++++++++
 tests/yacc-d-vpath.test              |   14 ++++
 tests/yacc-dist-nobuild.test         |    7 +-
 tests/yacc-subdir-objects-vpath.test |  137 ++++++++++++++++++++++++++++++++++
 tests/yaccdry.test                   |   39 ++++++++++
 tests/yaccdry2.test                  |   77 +++++++++++++++++++
 tests/yaccvpath.test                 |   12 +++
 13 files changed, 540 insertions(+), 3 deletions(-)
 create mode 100755 tests/yacc-cond-minimal.test
 create mode 100755 tests/yacc-cond.test
 create mode 100755 tests/yacc-subdir-objects-vpath.test
 create mode 100755 tests/yaccdry2.test

diff --git a/ChangeLog b/ChangeLog
index 4470c28..da291f3 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,22 @@
+2011-01-28   Stefano Lattarini  <address@hidden>
+
+       yacc: fix VPATH builds with FreeBSD make
+       Fixes automake bug#7884.
+       * lib/am/yacc.am (.BEGIN): New target, hack to support VPATH
+       builds with FreeBSD make.
+       * automake.in (lang_yacc_target_hook): Now update the private
+       make variable `am__yacc_c_outputs', used by the target above.
+       * tests/yacc-dist-nobuild.test: Update.
+       * tests/yaccdry.test: Extend.
+       * tests/yaccvpath.test: Likewise.
+       * tests/yacc-d-vpath.test: Likewise.
+       * tests/yacc-subdir-objects-vpath.test: New test.
+       * tests/yacc-cond.test: Likewise.
+       * tests/yacc-cond-minimal.test: New test, still failing with
+       BSD make.
+       * tests/yaccdry2.test: Likewise.
+       * tests/Makefile.am (TESTS): Update.
+
 2011-01-22   Stefano Lattarini  <address@hidden>
 
        configure: look for a yacc program to be used by the testsuite
diff --git a/automake.in b/automake.in
index fa458d6..c72e0bc 100755
--- a/automake.in
+++ b/automake.in
@@ -6082,6 +6082,13 @@ sub lang_yacc_target_hook
          }
       }
 
+    # Required to have FreeBSD make working correctly in VPATH builds.
+    # FIXME: It would be nice to add a yacc-generated C file to
+    # $(am__yacc_c_outputs) only if it's required by the build, e.g.
+    # only if it's not declared in a false conditional ...
+    Automake::Variable::define ('am__yacc_c_outputs', VAR_AUTOMAKE, '+',
+                                TRUE, $output, '', INTERNAL, VAR_PRETTY);
+
     if ($yflags_contains_minus_d)
       {
        (my $output_base = $output) =~ s/$KNOWN_EXTENSIONS_PATTERN$//;
diff --git a/lib/am/yacc.am b/lib/am/yacc.am
index 6d35cd4..8de0edc 100644
--- a/lib/am/yacc.am
+++ b/lib/am/yacc.am
@@ -46,3 +46,23 @@ endif %?MAINTAINER-MODE%
 ?!GENERIC??DIST_SOURCE?        $(am__skipyacc) \
 ## For non-suffix rules, we must emulate a VPATH search on %SOURCE%.
 ?!GENERIC?     $(SHELL) $(YLWRAP) `test -f '%SOURCE%' || echo 
'$(srcdir)/'`%SOURCE% y.tab.c %OBJ% y.tab.h %BASE%.h y.output %BASE%.output -- 
%COMPILE%
+
+## This is releated to automake bug#7884.
+## The automake-generated Makefiles usually distribute the yacc-derived C
+## files, so that the final user won't need a working yacc to build the
+## package.  But for some reason, in a VPATH build, FreeBSD make does not
+## see (or decides not to use) the yacc-derived and distributed C files
+## which are in the source directory, and tries to rebuild them (in the
+## build directory).  This might fail if the user hasn't a yacc program
+## installed, or even if he has one which is uncompatible with that used
+## by the developer.  So we use the following hack to ensure that, with
+## BSD make, the distributed, yacc-generated C sources are copied from
+## the source directory into the build directory, before starting the
+## "real" make operations.
+.BEGIN:
+       @for f in $(am__yacc_c_outputs); do \
+          if test ! -r $$f && test -f $(srcdir)/$$f; then \
+            echo " cp -p $(srcdir)/$$f $$f"; \
+            cp -p $(srcdir)/$$f $$f || exit 1; \
+          else :; fi; \
+       done;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index e639b6c..d0e3e31 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -807,8 +807,12 @@ yacc6.test \
 yacc7.test \
 yacc8.test \
 yaccdry.test \
+yaccdry2.test \
+yacc-cond.test \
+yacc-cond-minimal.test \
 yacc-dist-nobuild.test \
 yacc-nodist.test \
+yacc-subdir-objects-vpath.test \
 yaccpp.test \
 yaccvpath.test \
 yacc-d-vpath.test \
diff --git a/tests/Makefile.in b/tests/Makefile.in
index 25fc4a8..7eca925 100644
--- a/tests/Makefile.in
+++ b/tests/Makefile.in
@@ -1075,8 +1075,12 @@ yacc6.test \
 yacc7.test \
 yacc8.test \
 yaccdry.test \
+yaccdry2.test \
+yacc-cond.test \
+yacc-cond-minimal.test \
 yacc-dist-nobuild.test \
 yacc-nodist.test \
+yacc-subdir-objects-vpath.test \
 yaccpp.test \
 yaccvpath.test \
 yacc-d-vpath.test \
diff --git a/tests/yacc-cond-minimal.test b/tests/yacc-cond-minimal.test
new file mode 100755
index 0000000..2956f01
--- /dev/null
+++ b/tests/yacc-cond-minimal.test
@@ -0,0 +1,81 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that if a yacc-generated C source or header is not needed in the
+# build tree, make won't generate not copy it there.
+
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND], [:])
+AC_PROG_CC
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foo bar
+if COND
+foo_SOURCES = main.c
+else
+foo_SOURCES = parse.y main.c
+endif
+
+bar_SOURCES = $(foo_SOURCES)
+bar_YFLAGS = -d
+END
+
+cat > main.c << 'END'
+int main () { return 0; }
+END
+
+mkdir bin
+cat > bin/yacc <<'END'
+#!/bin/sh
+echo "$0 invoked, shouldn't happen!" >&2
+exit 1
+END
+cp bin/yacc bin/bison
+chmod a+x bin/yacc bin/bison
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH
+
+YACC=yacc BISON=bison
+export YACC BISON
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+mkdir build
+cd build
+../configure
+$MAKE
+test ! -r parse.c
+test ! -r bar-parse.c
+test ! -r bar-parse.h
+$MAKE clean
+$sleep
+: > ../parse.c
+: > ../bar-parse.c
+: > ../bar-parse.h
+$MAKE
+test ! -r parse.c
+test ! -r bar-parse.c
+test ! -r bar-parse.h
+
+:
diff --git a/tests/yacc-cond.test b/tests/yacc-cond.test
new file mode 100755
index 0000000..1ebce92
--- /dev/null
+++ b/tests/yacc-cond.test
@@ -0,0 +1,122 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check interaction of Yacc and conditionals.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+distdir=$me-1.0
+
+cat >> configure.in << 'END'
+AM_CONDITIONAL([COND], [:])
+AC_PROG_CC
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+bin_PROGRAMS = foobar zardoz
+
+AM_YFLAGS = -d
+
+if COND
+foobar_SOURCES = parse.y parse.h
+else
+foobar_SOURCES = parse.y main.c
+endif
+
+if COND
+zardoz_SOURCES = main2.c
+else
+zardoz_SOURCES = parse2.y parse2.h main2.c
+endif
+
+.PHONY: test-objs
+test-objs:
+       test -f parse.$(OBJEXT)
+       test ! -r parse2.$(OBJEXT)
+       test -f main2.$(OBJEXT)
+       test ! -r main.$(OBJEXT)
+END
+
+cat > parse2.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) {}
+%}
+%%
+foobar : 'f' 'o' 'o' 'b' 'a' 'r' {};
+%%
+END
+
+cat > main.c << 'END'
+int main (void) { return 0; }
+END
+
+cp main.c main2.c
+
+cat parse2.y main.c > parse.y
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+./configure
+$MAKE
+ls -l
+test -f parse.c
+test -f parse.h
+test ! -r parse2.c
+test ! -r parse2.h
+$MAKE test-objs
+
+$MAKE distdir
+ls -l $distdir
+test -f $distdir/parse.c
+test -f $distdir/parse.h
+# For all we know, the `parse2.[ch]' generated files might be required by
+# the end user (if he enable the COND conditional), so we must distribute
+# them.
+test -f $distdir/parse2.c
+test -f $distdir/parse2.h
+
+# Now check that yacc is not needed to build from a distribution tarball.
+
+chmod -R a-w $distdir
+
+mkdir bin
+cat > bin/yacc <<'END'
+#!/bin/sh
+echo "$0 invoked, shouldn't happen!" >&2
+exit 1
+END
+cp bin/yacc bin/bison
+chmod a+x bin/yacc bin/bison
+PATH=`pwd`/bin$PATH_SEPARATOR$PATH
+
+YACC=yacc BISON=bison
+export YACC BISON
+
+mkdir build
+cd build
+../$distdir/configure
+$MAKE
+$MAKE test-objs
+
+:
diff --git a/tests/yacc-d-vpath.test b/tests/yacc-d-vpath.test
index bb09990..4d2fed8 100755
--- a/tests/yacc-d-vpath.test
+++ b/tests/yacc-d-vpath.test
@@ -109,4 +109,18 @@ $MAKE distdir
 $FGREP maude $distdir/parse.c
 $FGREP MAUDE $distdir/parse.h
 
+# Now check that `make maintainer-clean' removes all yacc-derived
+# files (if any) created in the builddir.  Try it two times, to
+# be stricter in face of possible `.BEGIN' targets executed by BSD
+# make.
+cp config.status config.sav
+$MAKE maintainer-clean
+test ! -r parse.c
+test ! -r parse.h
+# Recreate Makefile, depfiles, etc.
+mv -f config.sav config.status; ./config.status
+$MAKE maintainer-clean
+test ! -r parse.c
+test ! -r parse.h
+
 :
diff --git a/tests/yacc-dist-nobuild.test b/tests/yacc-dist-nobuild.test
index 9061f57..26f5924 100755
--- a/tests/yacc-dist-nobuild.test
+++ b/tests/yacc-dist-nobuild.test
@@ -79,9 +79,10 @@ cd build
 $MAKE
 
 # Sanity check.
-chmod u+w ../$distdir
-rm -f ../$distdir/parse.c
-chmod a-w ../$distdir
+$sleep
+chmod u+w ../$distdir/parse.y
+echo >>  ../$distdir/parse.y
+chmod a-w ../$distdir/parse.y
 $MAKE >out 2>&1 && { cat out; Exit 1; }
 cat out
 $FGREP parse.c out
diff --git a/tests/yacc-subdir-objects-vpath.test 
b/tests/yacc-subdir-objects-vpath.test
new file mode 100755
index 0000000..c427e47
--- /dev/null
+++ b/tests/yacc-subdir-objects-vpath.test
@@ -0,0 +1,137 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Some tests for subdir parsers in VPATH builds.
+# See also test `yacc8.test'.
+
+required=yacc
+. ./defs || Exit 1
+
+set -e
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AM_PROG_CC_C_O
+AC_PROG_YACC
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AUTOMAKE_OPTIONS = subdir-objects
+bin_PROGRAMS = foo/bar baz/qux
+foo_bar_SOURCES = foo/parse.y
+baz_qux_SOURCES = baz/parse.y baz/main.c baz/parse.h
+AM_YFLAGS = -d
+
+.PHONY: test-com test1 test2
+test-com:
+       ls -l . foo $(srcdir) $(srcdir)/foo
+       test ! -r parse.h
+       test ! -r parse.c
+       test ! -r parse.$(OBJEXT)
+       test -f foo/parse.$(OBJEXT)
+       test -f baz/parse.$(OBJEXT)
+test1: test-com
+       test -f foo/parse.c
+       test -f foo/parse.h
+       test -f baz/parse.c
+       test -f baz/parse.h
+test2: test-com
+       test -f $(srcdir)/foo/parse.c
+       test -f $(srcdir)/foo/parse.h
+       test -f $(srcdir)/baz/parse.c
+       test -f $(srcdir)/baz/parse.h
+END
+
+mkdir foo
+cat > foo/parse.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) { return; }
+%}
+%%
+x : 'x' {};
+%%
+int main  (void) { return 0; }
+END
+
+mkdir baz
+cat > baz/parse.y << 'END'
+%{
+int yylex () { return 0; }
+void yyerror (char *s) { return; }
+%}
+%token BAZBAZ
+%%
+x : 'x' {};
+%%
+END
+cat > baz/main.c << 'END'
+#include "baz/parse.h"
+int main (void)
+{
+  return (BAZBAZ);
+}
+END
+
+$ACLOCAL
+$AUTOCONF
+$AUTOMAKE -a
+
+# Try a from-scratch VPATH build.
+mkdir sub1
+cd sub1
+../configure
+$MAKE
+$MAKE test1
+
+# Also, let's see if we can recover from removal of generated header
+# files.
+rm -f foo/parse.h baz/parse.h
+$MAKE foo/parse.h baz/parse.h
+test -f foo/parse.h
+test -f baz/parse.h
+$MAKE clean
+rm -f foo/parse.h baz/parse.h
+$MAKE
+test -f baz/parse.h
+# foo/parse.h is not declares in any *_SOURCES, nor #included by any
+# C file, so that it shouldn't be rebuilt by "make all".
+test ! -r foo/parse.h
+
+cd ..
+
+# Generate yacc-derived files with an in-tree build.
+./configure
+$MAKE foo/parse.c foo/parse.h baz/parse.c baz/parse.h
+$MAKE distclean
+ls -l . foo
+test -f foo/parse.c
+test -f foo/parse.h
+test -f baz/parse.c
+test -f baz/parse.h
+
+# Now try a VPATH build with yacc-generated files already present in
+# the srcdir.
+YACC=false BISON=false
+export YACC BISON
+mkdir sub2
+cd sub2
+../configure
+$MAKE
+$MAKE test2
+
+:
diff --git a/tests/yaccdry.test b/tests/yaccdry.test
index d30fb80..8eebc67 100755
--- a/tests/yaccdry.test
+++ b/tests/yaccdry.test
@@ -21,6 +21,8 @@ required=yacc
 
 set -e
 
+distdir=$me-1.0
+
 cat >> configure.in << 'END'
 AC_PROG_CC
 AC_PROG_YACC
@@ -49,12 +51,49 @@ END
 $ACLOCAL
 $AUTOMAKE --add-missing
 $AUTOCONF
+
+# Try an in-tree build first.
+
 ./configure
 $MAKE
 
 rm -f parse.h
+
+$MAKE -n
+test -f parse.c
+test ! -f parse.h
+
 $MAKE -n parse.h
 test -f parse.c
 test ! -f parse.h
 
+# Now try VPATH build.
+
+$MAKE distdir
+mkdir build
+cd build
+../$distdir/configure
+
+$MAKE -n
+test ! -f parse.c
+test ! -f parse.h
+
+$MAKE -n parse.h
+test ! -f parse.c
+test ! -f parse.h
+
+rm -f ../$distdir/parse.h
+
+$MAKE -n
+test ! -f parse.c
+test ! -f parse.h
+test ! -f ../$distdir/parse.h
+test -f ../$distdir/parse.c
+
+$MAKE -n parse.h
+test ! -f parse.c
+test ! -f parse.h
+test ! -f ../$distdir/parse.h
+test -f ../$distdir/parse.c
+
 :
diff --git a/tests/yaccdry2.test b/tests/yaccdry2.test
new file mode 100755
index 0000000..22902af
--- /dev/null
+++ b/tests/yaccdry2.test
@@ -0,0 +1,77 @@
+#! /bin/sh
+# Copyright (C) 2011 Free Software Foundation, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2, or (at your option)
+# any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+# Check that, in a VPATH build, no yacc-derived C file is created in
+# the build directory by targets like "make clean".
+
+. ./defs || Exit 1
+
+set -e
+
+unset YACC || :
+
+distdir=$me-1.0
+
+cat > my-yacc <<'END'
+#!/bin/sh
+echo 'int main () { return 0; }' > y.tab.c
+: > y.tab.h
+END
+chmod a+x my-yacc
+
+cat >> configure.in << 'END'
+AC_PROG_CC
+AC_SUBST([YACC], ['$(abs_builddir)/my-yacc'])
+AC_OUTPUT
+END
+
+cat > Makefile.am << 'END'
+AM_YFLAGS = -d
+bin_PROGRAMS = foo
+foo_SOURCES = parse.y
+END
+
+: > parse.y
+
+$ACLOCAL
+$AUTOMAKE --add-missing
+$AUTOCONF
+
+# From-scratch VPATH build.
+mkdir build-scratch
+cd build-scratch
+../configure
+for target in mostlyclean clean distclean; do
+  $MAKE $target
+  test ! -f parse.c
+  test ! -f parse.h
+done
+
+cd ..
+./configure
+$MAKE distdir
+
+# VPATH build from distributed tarball.
+mkdir build-dist
+cd build-dist
+../$distdir/configure
+for target in mostlyclean clean distclean; do
+  $MAKE $target
+  test ! -f parse.c
+  test ! -f parse.h
+done
+
+:
diff --git a/tests/yaccvpath.test b/tests/yaccvpath.test
index 52092fa..853c5df 100755
--- a/tests/yaccvpath.test
+++ b/tests/yaccvpath.test
@@ -99,4 +99,16 @@ END
 $MAKE distdir
 $FGREP maude $distdir/parse.c
 
+# Now check that `make maintainer-clean' removes all yacc-derived
+# files (if any) created in the builddir.  Try it two times, to
+# be stricter in face of possible `.BEGIN' targets executed by BSD
+# make.
+cp config.status config.sav
+$MAKE maintainer-clean
+test ! -r parse.c
+# Recreate Makefile, depfiles, etc.
+mv -f config.sav config.status; ./config.status
+$MAKE maintainer-clean
+test ! -r parse.c
+
 :
-- 
1.7.2.3