Re: [PATCH 01/10] tests: migrate legacy tests/cdemo tests to Autotest.

[Gary V. Vaughan]

On 25 Nov 2011, at 18:59, Stefano Lattarini wrote:
> Hi Gary.

Hi Stefano,

> On Friday 25 November 2011, Gary V wrote:
>> The best reason I can find for keeping the various demo directories
>> around (despite the fact we already make use of the much better test
>> harness of Autotest for all our new test cases) from the last time
>> I wanted to migrate everything out of the legacy testsuite, was that
>> it exercises the distribution manager's autotools combination on the
>> testers machine, where the Autotests always use the users autotools.
>> That's no argument as far as I can see: we want tests to fail as
>> early as possible on a users machine to help him know things are not
>> going to work properly if he keeps going - and having the legacy
>> suite pass is only encouraging users to fight with broken installs.
>> 
>> This series of patches migrates all 9 of the demo directory test
>> groups into Autotest, and allows us to remove most of the legacy
>> testsuite cruft at the end.
>> 
> Just my 2 cents: I'd like to see the test scripts converted one at
> the time, rather than one group at the time (and assuming that this
> is feasible), as I found your huge patches basically un-reviewable
> in their present state.

The legacy tests are in sets that can't be broken apart without leaving
the tree in an inconsistent state part way through that set.

I could break them up a little more tho', if you think that would help,
so instead of converting one demo directory all at once, then a final
patch to clean out the configury cruft... there'd be 9 sets of patches
each containing almost everything in the current patch, except the
deletion of the the files in the given test/demo directory, followed by
a series of tiny patches each adding a dozen lines like this:

+## ----------- ##
+## Mdemo conf. ##
+## ----------- ##
+
+AT_SETUP([ltdl load shared and static modules])
+
+_LT_SETUP
+
+_LT_CHECK_CONFIG([], [^build_old_libs=yes], [^build_libtool_libs=yes])
+_LT_CHECK_EXECUTE
+_LT_CHECK_INSTALL
+_LT_CHECK_UNINSTALL
+
+AT_CLEANUP

plus removing the equivalent legacy set of 3 *.test files, and then a
final patch to delete the given test/demo tree, and relevant lines from
Makefile.am.

It'll take me a while to go through and do that, and we'll end up with
10 large patches each setting up a new tests/demo.at file, about 25
tiny patches per the above, then 10 small patches each removing one of
the tests/demo trees, and then the final cruft cleanout patch unchanged.

If that will make a big difference, let me know and I'll retract these
patches and post 50 or so to replace them in a week or two when I've
gone through and broken out the tiny per-test-group sets per the above.

>> There's still a few legacy tests
>> left at the end, which I'll tackle later, but at least maintenance
>> is a whole lot easier now that we don't need to wait for 9 additional
>> directories to autoreconf every time we run bootstrap, or distcheck,
>> or roll up a distribution tarball to test on the local network.
>> 
>> This is all in keeping with the theme of most of the patches I've
>> posted this year, to make libtool easier and more fun to maintain
>> and contribute to, in the hope of getting more people involved.
>> 
>> As usual, subject to feedback, I'll push this whole series in
>> 72 hours or so.  Make distcheck passes for me on my Mac 10.7 and
>> my Arch Linux x86_64 machines, but it would be great if folks with
>> access to other machines could give it a spin to see whether I
>> broke any of the tests during migration... if you'd like a pre-
>> rolled distro with my pending patches applied to do that, then
>> please do ask.
>> 
> If you want to send me such a tarball, I might run the testsuite on
> Solaris 10, Cygwin 1.5.25, NetBSD 5.1 and OpenBSD 4.6 at least.

Much appreciated.  Tarballs here:

  http://vaughan.pe/libtool/libtool-2.4.2.135-aa59c.tar.gz
  http://vaughan.pe/libtool/libtool-2.4.2.135-aa59c.tar.xz

>  But
> then you should allow for more than three days for sending feedback
> -- at least a week.

That's fine, and running on those arches would be a great help in
giving me confidence the migration didn't break anything.

It'll take me at least a week to redo everything into smaller pieces
anyway... (unless you think the time spent here will not make much
difference to your review). But do let me know either way :)  And
thanks also for offering to make the time to look them over.

Cheers,
-- 
Gary V. Vaughan (gary AT gnu DOT org)