memory allocation (was: Re: Dynamic objects)

[Paul Smith]

On Mon, 2013-04-29 at 19:30 +0100, Tim Murphy wrote:
> I must clarify - I think that make should provide plugins with an
> allocation mechanism. Not the other way around.

It's probably a good idea for make to provide a "gmk_free()" function
that will free memory returned to the plugin when it calls gmk_*()
functions such as gmk_expand().  Is that sufficient to deal with this
problem?

> the snprintf model for dealing with expansion is not so bad - I mean
> the problem is that nobody knows how big an expansion is going to be
> in the end, right?  So how does make deal with this already? The same
> way would be fine for the plugin and it would be nice to not simply
> push that problem on to all plugin writers.

make calls malloc() and if the buffer is not big enough it calls
realloc().  But the problem is that while make is expanding and
allocating, it's also interpreting.  And that interpretation might have
side-effects.

Suppose, for example, we enhanced gmk_expand() to take a buffer and a
plugin invoked:

    gmk_expand(buf, buflen, "$(info expanding) $(FOO)");

This will return the expansion of the FOO variable, but it will ALSO
print "expanding" as the result of calling the info function.  Now
suppose that the result of expanding $(FOO) was too large to fit into
buf, so the function returns the length needed and the plugin
reallocates the buffer to be large enough and re-invokes gmk_expand()...
now it will print "expanding" AGAIN.

Of course the side-effect might not be so innocuous as double-printing.

On Mon, 2013-04-29 at 22:34 +0300, Eli Zaretskii wrote:
> At least on Windows, it can be a real problem, because the libraries
> with which a shared object was linked are hardcoded into it, and
> there's more than one way of allocating memory.
> 
> How about a callback for allocating memory?  Then Make could call that
> callback and get memory that the extension could free.

This could work, at the cost of an extra allocation and buffer copy for
each invocation of gmk_expand() (etc.)  We would basically call our
normal expand and get back a buffer allocated by us, then if there was a
separate allocator registered we'd call that to get enough memory to
hold the buffer, then copy over from our buffer to theirs, then free our
buffer and pass back theirs.

If the method at the beginning of this email (providing a gmk_free()
function that will free memory returned from gmk_expand()) is
sufficient, though, wouldn't that be a better/more efficient solution?

As long as we don't have one side (between make and the object)
allocating memory and the other side freeing it, is that enough?