Re: make dist improvements

[Behdad Esfahbod]

Ralf Wildenhues wrote:
> Hi Behdad,
> 
> thank you for you helpful feedback!  Addressing parts of it:

Hi,

Thanks again for the reply.


> * Behdad Esfahbod wrote on Fri, Sep 19, 2008 at 01:22:54AM CEST:
>> Also may be a good idea to instruct tar to not record uid of the user making
>> the tarball?
> 
> Is there any way of doing that portably?  Otherwise, the only choice I
> see is to factorize and do it for GNU tar only.

Not that I know of.  I can't even find how to do that with GNU tar.  But
making tar understand some env var like the GZIP=--best automake already does
sounds easy enough, even if it only produces results in a few years. :)

>> - Parallel make:
>>
>> A fellow maintainer had weird problems making a snapshot today, which I 
>> turned
>> out to be caused by him having MAKEFLAGS=-j2.  Trying to fix that I found 
>> that
>> automake can be a bit more helpful.
> 
> Agreed.  This is a problem.
> 
>> Anyway, what would be helpful is if dist-gzip et al (distcheck also?) did
>> locking as described at the end of automake manual section on Handling Tools
>> that Produce Many Outputs.  That way the implied dist in distcheck and the
>> dist my release-upload target will require will not race and dist rules will
>> be run just once.
> 
> I've tried to wrap my head around this, but I can't seem to get it.
> The issue here is different; the 'distdir' rule doesn't generate
> multiple outputs, at least not conceptually.  The 'dist' rule does,
> but that bit is not your problem AFAICS.  The issue is that all those
> dist-TYPE rules remove $(distdir) at the end.  If you get more than one
> of those rules to run, you have a problem, with or without parallel
> make.  Locking won't help either.  With GNU make, the rules could
> enquire whether other dist-TYPE rules are yet to be run, and only remove
> $(distdir) at the end, but I don't see any way to do this in portably.

Yeah, I spent half an hour right now trying to write something but it gets too
messy too soon.  I was thinking like "each of dist-* will touch a file in a
lock dir and then wait for the first process to finish building distdir.  when
done, each dist-* removes the file it touched and if it succeeds removing the
lock dir it will also remove the distdir."  But doing it properly doesn't
quite fit my head.
                

>> The scenario at hand is that we have some make targets to make releasing
>> easier.  In particular:
>>
>> release-check: \
>>         release-verify-sane-changelogs \
>>         release-verify-sane-tests \
>>         release-verify-even-micro \
>>         release-verify-newer \
>>         release-remove-old \
>>         distcheck
>>
>> release-upload: release-check $(tar_file) $(sha1_file) $(gpg_file)
> 
> Just let this rule depend on 'dist', rather than on '$(tar_file)'.

Yeah, but that would have caused dist and distcheck interfering.

>>         mkdir -p releases
> 
> $(MKDIR_P) (when using Automake 1.10+), or $(mkdir_p) (otherwise).

Thanks.

>> The problem is that my release-upload rule relies on release-check (by way of
>> distcheck) create $(tar_file), which is my fault of course.
> 
> Hmm.  So you want locking between 'dist' and 'distcheck'.  Same problem
> as above, IIUC.

Yeah.

> If somebody sees a way out, I'd be interested to hear it.
> 
> Cheers,
> Ralf


behdad