automake-patches
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: TAP support in automake (was: Re: automake 1.11.3 check-TESTS and co


From: Bob Friesenhahn
Subject: Re: TAP support in automake (was: Re: automake 1.11.3 check-TESTS and command line length)
Date: Sat, 31 Mar 2012 15:23:49 -0500 (CDT)
User-agent: Alpine 2.01 (GSO 1266 2009-07-14)

On Sat, 31 Mar 2012, Stefano Lattarini wrote:

Well, automake being automake, it only depends on portable awk[1] ;-)
OK, portable "nawk" actually (as found by the autoconf builtin macro
AC_PROG_AWK), but I don't know of any non-museum machine lacking that.

Good.

The new feature is already documented in the automake manual on the
'master' branch; and this documentation should be release-quality
already.  But maybe I should make this new feature more visible before
the 1.12 release?  If yes, how do you suggest to do so?

Certainly mentioning and discussing it again on the 'automake' list (happening now) is useful. The previous discussion I found was from almost a year ago before development of the feature was started.

As for the implementation quality, there's still an annoying glitch in
that you need to *manually* copy the automake-provided 'tap-driver.sh'
script yourself into your package[2] -- "automake --add-missing" won't
work -- and define some variables by hand; all of this is documented
in the manual, and should hopefully be fixed for automake 1.12.1.

This does not seem like a big problem.

Russ Allbery's C TAP Harness looks reasonable as well. It also requires adding a single file (.c) to the source tree and a rule to build it as a test program.

The philosophy of TAP is rather different than Automake's existing test suite. These differences may prove challenging when switching from Automake tests to TAP:

  * Every TAP test needs to somehow know its unique test number (so it
    can print a result "ok 35").  If the tests are all in the same
    module then they can simply increment an integer but if they are
    not then there is a problem (or they can always plan for one
    test).

  * TAP tests need to know if they are expected to pass or fail.

A major feature of how GraphicsMagick is using the Automake tests framework is that it determines which tests are XFAIL based on what features were configured into the software. The list of tests which are expected to fail is produced based on Automake conditionals. It is useful for the user to see tests XFAIL as a reminder that the configuration may be missing something they wanted. How is this best handled for TAP tests?

Bob
--
Bob Friesenhahn
address@hidden, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/



reply via email to

[Prev in Thread] Current Thread [Next in Thread]