Re: serial-tests option and backwards compatibility

[Bob Friesenhahn]

On Thu, 17 Jan 2013, Stefano Lattarini wrote:
> > (I know future is to use TAP log protocol, but that's not an excuse
> > to break working package tests.)
> Actually, TAP makes sense in a lot of situations, but not always,
> an maybe not even most of the time.  First, several TAP tests in
> a single script are usually run serially (most TAP-producing libraries
> out there are not multi-threaded), and on fast and many-cores machines
> this can be a real performance killer if you implement your testsuite
> in just a handful of TAP scripts.  In addition, TAP is overkill for
> many testing scenarios, not worth the trouble.  The good thing of the
> parallel Automake harness is that it allows you to freely mix "classic"
> test scripts and TAP based test script, allowing you to use the best
> of both worlds (Automake' own test suite makes an heavy use of this
> capability).

My own experience with TAP is that the per-test overhead was reduced 
so that the tests actually ran considerably faster (particularly under 
Microsoft Windows).  If your test software runs really slow, then the 
experience may be different.  There is a lot of overhead in the 
parallel-tests framework.

For parallel builds, just make sure that you have enough independent 
TAP test scripts to keep the system busy when running parallel tests. 
Only very CPU-bound single-threaded tests will be sped-up much by the 
parallel-tests framework.  If your software is already threaded, then 
parallel-tests may lengthen the test time due to creating too many 
threads on the system.

Of course, TAP does not solve the backward-compatibility problem.

Bob
--
Bob Friesenhahn
address@hidden, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/