Re: Cached variables

[Ramón García]

I am thinking how to implement the model of persistent variables that
you are proposing. I don't see a clear way of saying, if the value in
cache of FOO is different from md5sum of prerequisite, then store the
new value of foo after a successful compilation (see bellow why).
Also note that the presistent value of the MD5 sum is different for
each pair (target, prerequisite). If you don't think that it has to be
stored separatedly for each different target of some prerequisite, see
that the user can compile one prerequisite in one invocation of make,
and so the last used value of a prerequisite may be different for
different targets.

In addition, a different persistent variable is needed for each
prerequisite of each target. So a syntax like:

target: cache variable = value

is not enough. Otherwise, how would it be posible that .OUT_OF_DATE
evaluates? Remember that we decided that .OUT_OF_DATE is evaluated for
each pair (target,prerequisite) for simplicity.

The issues that should be solved are:
1) Need a different variable for each (target,prequisite) pair, for
implementing content-change based dependencies.
2) Need a builtin function so specify: if this expression is different
from the stored value, then change the stored value after the next
successful compilation. Perhaps, I miss something, but I can't see how
to perform this without a new function.

I talked about introducing a keyword "cached", but for something
different: avoiding repeated evaluation of MD5 sums and similar.

Let me tell you the method that I have programmed for persistent variables.

Whatever model is used, note some of the requirements of the problem.
The persistent variable should be saved after a successful compilation
of its target. Otherwise, necessary compilations of the target will be
skipped, if the previous compilation failed or was interrupted by the
user. For instance, if a MD5 sum is used to trace content changes, the
sum saved as last used should be saved after the compilation with that
file was successful, otherwise, this compilation should be repeated,
and thus this sum should not be stored as last sum used for
compilation.

This method designed specifically for checking if something changed
since the last successful compilation of a target. The builtin
function changed-value, that returns true if a given value changed
from the last call made before the last successful compilation. The
syntax is

${changed-value target,name,value}

That is, when one calls changed-value, the association of name and
value is saved in a hash table of target, and then, if the compilation
is successful, it is recorded to the state file. In addition, it
returns true depending on whether the last stored value is equal to
this one.

Note that this is almost equal to your initial proposal for the
meaning of .OUT_OF_DATE. A simple example would be:

a: b c
a: .OUT_OF_DATE=${changed-value $@,$<,${shell md5sum $<}}

This is inefficient in the case the same prerequisite is used for
several targets. I was thinking in using "cache" keyword for solving
exactly this problem.

The state file is the directory of the target + dot + the name of the
target + encoded name of the prerequisite (encoding any path
separating characters). The rationale is to use a name that depends
only on the name of the target and the prerequisite, and that usually
exists, since it is reasonable to assume that the directory of the
target is writable.

The model of changed-value for persistent variables is inflexible, in
that it is specifically designed for the case of storing a value for
checking if it is the same used in the last compilation. This was
choosen for easy of use. Of course, different models, more flexible or
more rigid are posible.

Finally thanks for your effort during your vacations. If you prefer to
rest, we can wait until you finish your vacations. Otherwise, we can
communicate by SMS.

On 7/22/07, Paul D. Smith <address@hidden> wrote:
> Hi Ramon;
>
> I'm on vacation for another week so I'll have limited email access.  I
> also don't have access to your previous email about caching variable
> values, as it's on my system at home.
>
> However, I do like the idea as a way to introduce state into GNU
> make.  It seems very useful and very flexible.
>
> I prefer the keyword model, where "cache" is another keyword that has
> all the same characteristics (syntactically) as "export".  So you
> could use:
>
>       cache FOO = bar
>
> or:
>
>       FOO = bar
>       cache FOO
>
> Also something like this would be valid:
>
>      foo: cache FOO = bar
>
> The output cache file can simply be a makefile with variable settings
> in it; we can choose a default name for it (something like
> ./.makesave.mk or whatever... it would be nice to keep it an 8.3 name
> so it works on the DOS port etc.)
>
> One tricky bit here is allowing for an "initial value"; if the
> variable is not set in the cache then what value will it have?
>
> One idea is to say that this:
>
>     cache FOO = bar
>
> means that if FOO doesn't exist in the cache, it will be set to bar.
> If it does, it will have the value in the cache.  In this model, this:
>
>    cache FOO = bar
>
>    foo: FOO = zee
>
> means that FOO will always be "zee" in target "foo", regardless of
> what's in the cache.  Also this:
>
>    FOO = bar
>    cache FOO
>
> now doesn't make much sense because the value of FOO will always be
> "bar" regardless of what's in the cache.
>
> I think we'll need some example use cases here to flesh out how this
> will work.
>
> --
> -------------------------------------------------------------------------------
>  Paul D. Smith <address@hidden>          Find some GNU make tips at:
>  http://www.gnu.org                      http://make.paulandlesley.org
>  "Please remain calm...I may be mad, but I am a professional." --Mad Scientist