[bug #40610] gmake 4.0 loops rebuilding included makefiles

[Paul D. Smith]

Follow-up Comment #3, bug #40610 (project make):

Yes, thanks, I meant _both_ put the mkdir inside the depend.mk rule _and_
remove the prerequisite (as in my example).  Sorry for not being clear.

I don't believe this is a bug in GNU make; this is a sub-optimal makefile,
combined with what seems to be a bug, or at least non-standard behavior, in
the tmpfs handling of directory timestamps.

It's almost never correct to list a directory as a prerequisite of a target. 
This is because the time-last-modified operation of a directory is completely
different than that of a regular file: the TLM of a directory is changed
whenever a new file is added, removed, or renamed.  So if a target depends on
a directory the target will be considered out of date whenever ANY file in
that directory is added, removed, or renamed.  Very unlikely to be what you
want.

The first time you run make it creates objdir at time T, then runs the
depend.mk rule which removes and recreates depend.mk (which updates the objdir
timestamp to T+X), then writes to depend.mk and closes it which sets the
depend.mk timestamp to T+X+Y.  Then we re-exec, depend.mk's time of T+X+Y is
greater than objdir's time of T+X, and all is well.

If you run it again where depend.mk doesn't exist or is older than objdir
(maybe someone added a file to objdir), then depend.mk will be removed (due to
the rm) then created (modifying objdir's timestamp), written, and closed
again, which will update the timestamp of depend.mk.  Again here, depend.mk
_should_ have a newer timestamp than objdir.

In this case somehow the timestamp of objdir is always newer than the
timestamp of depend.mk.  That says to me that the handling of directory
timestamps in this implementation of tmpfs is non-standard in some way, so
that for some reason it's updated again after the depend.mk file is closed.

I'm not sure why it wasn't triggered with GNU make 3.82; perhaps, as you say,
for some reason that version of make was not using sub-second timestamps and
this version is (I don't know why that would be however).

Some things you can try are first, don't use the "rm"; this will keep the
objdir timestamp from being updated, because opening an existing file (with
truncation), writing to it, and closing it should not change the directory
timestamp:


objdir/depend.mk: objdir
        echo 'foo: bar' > objdir/depend.mk


This is just an experiment, really.  I don't suggest this is a good solution
as it's too obscure.

The second (more correct) thing to try is order-only prerequisites instead of
normal prerequisites:


objdir/depend.mk : | objdir
        rm -f objdir/depend.mk
        echo 'foo: bar' > objdir/depend.mk


This ensures that the objdir rule is run first, but it also does not rebuild
depend.mk if objdir has a newer timestamp.

Please let me know how these two modifications work.

As for the infinite looping, ideally we could avoid that.  However, the
behavior of make when it detects that a makefile is remade is to re-exec
itself (with exec(2)).  This means that a new instance of make completely
replaces the existing instance: there is no state that's transferred from the
original to the re-exec'd copy.  So the re-exec'd copy has no way to know that
it already built that file.  If the file is out of date still, make will build
it again.

At one point I had experimented with adding -o <file> arguments to the re-exec
so that the new make would not rebuild something we just rebuilt.  The major
problem with this is that sometimes there were so many arguments added this
way that we exceeded the environment limit and the re-exec operation failed.

At some point I'd like to implement a "read options from a file" capability
for GNU make, as other tools are starting to do, and when that happens I may
be able to revisit this decision since we'll have a virtually unlimited number
of options available.  Until then, this is just an issue we have to live with
and be aware of when writing makefiles that recreate themselves: we need to be
careful to avoid infinite recursion.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/bugs/?40610>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/