a saner bootstrap script [Was Re: I fixed this once already]

[Gary V. Vaughan]

Anyone:

It's beginning to look as though all this work is, once again, in very
real danger of just slipping quietly through the cracks.

On 8 Sep 2011, at 01:56, Gary V. Vaughan wrote:
> Bumping this thread back to the top of the pile for it's 1 month 
> anniversary...
> 
> On 24 Aug 2011, at 00:51, Gary V. Vaughan wrote:
>> Hi Paul,
>> 
>> Just want to keep this on your radar...
>> 
>> Also, please note that I've made some small improvements to the script,
>> and pushed to the GNU Zile repository, but not the others listed below.
>> If you'd like me to synchronize before you review, please ask.
>> 
>> On 15 Aug 2011, at 09:29, "Gary V. Vaughan" <address@hidden> wrote:
>>> On 15 Aug 2011, at 04:19, Paul Eggert wrote:
>>>> On 08/14/2011 01:38 PM, Gary V. Vaughan wrote:
>>>>> Please tell me to stop bugging the list if you're tired of my
>>>>> mentioning this from time to time. On the other hand, if you just
>>>>> need more proof that this one is working as well as I say, I'll add
>>>>> it on a topic branch to the gnulib using projects I have commit
>>>>> rights to, and update the bootstrap.confs I wrote for them last
>>>>> year provided someone will take a look afterwards, with the intention
>>>>> of agreeing to supercede the existing bootstrap if everything works
>>>>> as advertised.
>>>> 
>>>> I haven't had time to look at the complete rewrite, but I think
>>>> this is a good way to proceed.  I don't recall which projects
>>>> you were doing.
>>> 
>>> Great! Thank you :)


Is there anything I can do to save it?  Would it help if I run the new
bootstrap scripts in each of the projects I listed earlier in this thread
again, pasting the output to the list?  What about if I offer to use
my commit bit to swap the bootstrap implementation myself at the end of
the month if no-one has asked me not to before then?  Or next month?
I've waited a year already, so a few more weeks is hardly significant,
as long as it is facilitates some actual concrete progress.

I know it's difficult to review a rewrite, but you can't meaningfully
present a total rewrite of anything significant as a series of logical
patches with the net effect of removing an old file entirely and
replacing it with something completely different.

I also know that it's not low-hanging fruit, because the existing script
is limping along okay (at least for as long as potential users who look
inside, but then turn their noses up when they can't figure out how to
make it work with their project, and so just ignore the existing script
use something else instead).

I would say that it's clear at this point that no-one has the right
combination of motivation and time to review the script properly (and
actually, what does that really mean anyway?  Would you just try to
use it in place of the existing script in a bunch of projects that have
different bootstrap processes -- a bit like the selection I present in
the quoted email below, say).  I have already tested everything
extensively and quite rigorously.

Is there really any harm in just adopting this much improved script and
dealing with the tiny amount of fallout that might result?  The worst
case scenario I can come up with is that my bootstrap rewrite turns out
to be a total pile of crap, and a ton of important gnulib clients try
to migrate to it, but it doesn't work, so they send hatemail to this
list and happily carry on using whatever they had before.  Gnulib calls
me an idiot, and reverts the patch that swapped out the bootstrap
implementation.  Across the entire GNU development environment, I may
have wasted, at most, a handful of developer-hours.

In reality, this replacement (which took several developer-weeks for
me to design, implement, test and debug... and has been enjoying micro-
improvements and stabilisation patches for a year already!!) is proving
very reliable and useful in real software right now.  And it is plainly
a more carefully designed and implemented script that what gnulib is
currently wearing.  I expect the real impact of adopting it will be
closer to "huh, my bootstrap script just got bigger" with a sprinkling
of "cool, I can actually understand this well enough to extend my bootstrap
process a little!".

So, please please, would someone at least give it a shot so everyone
else has a chance at enjoying it too?  Or at least would you make it so
that gnulib-tool gives out the url to my version for folks who casually
import the gnulib bootstrap unaware that there is a much cleaner, faster,
and understandable reimplementation which has been heavily tested in a
huge number of bootstrap scenarios easily available outside gnulib itself?
I would be amazed if anyone who takes the time to deliberately choose one
implementation over the other would pick the existing gnulib implementation.

And let's not forget, that I actually documented how to extend my
implementation in a fair amount of detail, so the architecture is
completely transparent, making it mostly immune to turning into another
aggregate of whatever cruft other projects might donate to make their
own bootstrap.conf a little smaller.

I'm treating the bootstrap script in GNU Zile git as the master copy
if you would like to perform any review or testing... I'll be happy to
make the time to update all the github repos I set up for review purposes
if that will help, but I think nothing substantial differs since I last
updated everything.

Hoping for a response this time...

On 8 Sep 2011, at 01:56, Gary V. Vaughan wrote:
> On 24 Aug 2011, at 00:51, Gary V. Vaughan wrote:
>> 
>>> Here's what I have so far:
>>> 
>>> * address@hidden:gvvaughan/GNU-bison.git in gary/bootstrap
>>> https://github.com/gvvaughan/GNU-bison/commits/gary/bootstrap
>>> 
>>> The bison bootstrap is very straight-forward, so I just redid
>>> it with my new bootstrap script in the gary/bootstrap branch. I
>>> don't have a commit bit for bison, so I've put a mirror with
>>> my branch in it up on github, per the header to this bullet.
>>> 
>>> * address@hidden:gvvaughan/GNU-coreutils.git in gary/bootstrap
>>> https://github.com/gvvaughan/GNU-coreutils/commits/gary/bootstrap
>>> 
>>> I can't get the currently checked in bootstrap to finish running
>>> on current master, which makes porting it's contents into the
>>> new bootstrap.conf futile for me. Instead I've updated to the new
>>> bootstrap script on the original branch I made when I was writing
>>> it, and pushed a mirror to github here too. (Note that I was having
>>>  problems compiling sort just after threads had been added around
>>> that time, but a `make -k' completes on my Mac OS 10.7 machine.)
>>> 
>>> This one is interesting because it uses gettext and also has a
>>> fairly torturous bootstrap process.  I wasn't able to eliminate
>>> slurp entirely, but there is only something much much smaller and
>>> possible to understand remaining in bootstrap.conf.
>>> 
>>> * address@hidden:gvvaughan/GNU-libtool.git in topic/use-gnulib
>>> https://github.com/gvvaughan/GNU-libtool/commits/topic/use-gnulib
>>> 
>>> I'm not ready to push my working branches up to savannah yet, since
>>> I might want to rebase them locally first.  So I've put a mirror up
>>> in my github account for you to try out.
>>> 
>>> This one is interesting, because Libtool uses a complicated bootstrap
>>> process, with several subprojects to autoconfiscate, and additional
>>> options to bootstrap itself added using the extension mechanisms alone.
>>> You'll notice in configure.ac that there's a good deal of mucking
>>> about with M4 macros around the values that bootstrap is still
>>> extracting quite successfully.
>>> 
>>> * address@hidden:gvvaughan/GNU-m4.git in gary/bootstrap
>>> https://github.com/gvvaughan/GNU-m4/commits/gary/bootstrap
>>> 
>>> Again, I've mirrored some private branches that are subject to
>>> rebasing before merging and committing back to the savannah repo
>>> so that you can see what is going on here. But, I have rebased this
>>> on onto the HEAD of branch-1.4, with the latest gnulib in a submodule.
>>> I haven't updated the other branches yet (branch-1.6 and master), but
>>> that will happen automatically once gnulib is presenting the new
>>> bootstrap script.
>>> 
>>> M4 is interesting because it uses "the other" methodology of treating
>>> gnulib-cache.m4 as truth and checking it in, with bootstrap managing
>>> it rather than creating it.  And it uses git-version-gen to generate
>>> its version number on the fly, which bootstrap copes with quite easily.
>>> You can also try out the --skip-git and skip-po options here. The new
>>> bootstrap script started life because the current one has (had?) a
>>> few incongruencies that prevented it working in sympathy with how Eric
>>> and I manage M4 development.
>>> 
>>> * git://git.savannah.gnu.org/zile.git in topic/sane-bootstrap
>>> http://git.savannah.gnu.org/cgit/zile.git/log/?h=topic/sane-bootstrap
>>> 
>>> This branch (with my bootstrap script in it) has already been merged
>>> to master and lua branches, both of which demonstrate very different
>>> bootstraps (especially lua, which has no compiled code in it at all!).
>>> 
>>> * It seems I also made a start at converting GNU tar to my new bootstrap,
>>> but I came unstuck when porting the copy_files function out of the
>>> existing bootstrap into a new bootstrap.conf.  I don't really have time
>>> to finish it now, but I hope the other projects above give you enough
>>> confidence to adopt the new script directly into gnulib for propagation
>>> into other projects, without me needing to finish the GNU tar conversion
>>> on my own first?
>>> 
>>> On 15 Aug 2011, at 04:19, Paul Eggert wrote:
>>>> I could also try it out with a smaller project
>>>> that isn't near a release (diffutils comes to mind ...).
>>> 
>>> Please do, that would be awesome.  The documentation is in the tarball
>>> attached earlier in the thread, I didn't bother to add another copy to
>>> every repo above... although the script is well commented and easy to
>>> understand even without the documentation.
>>> 
>>> Thanks again for taking the time to follow through on this.


Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)