--- Begin Message ---
|
Subject: |
Various issues with the test suite framework |
|
Date: |
Fri, 20 Apr 2012 13:13:58 +0200 |
Hi!
I have seen that check-html will be removed, or rather moved
into the contrib part, but there are a few issues:
- the target is not declared recursive, so one has to write
the bouncing target herself.
- because of that, the "naive" implementation of check-html
that just bounces the right directories will miss the
dependencies that check features (in other words, make
check properly recursively builds before, while check-html
would just go into the typical tests/ directory, and then
bounce to "check", so it completely missed the recursion
in the other directories and their dependencies).
- RECURSIVE_TARGETS cannot be highjacked by the users, that's
sad. Could a $(USER_RECURSIVE_TARGETS) be added to
RECURSIVE_TARGETS so that we can add our own targets that
traverse the whole package? Actually maybe this should
be an Automake option to add recursive targets, and
then TARGET-am and TARGET-recursive would be generated and
bound to the proper TARGET.
- the .log.html recipe features $$RST2HTML, but in '', so it is
useless. And anyway $(RST2HTML) already does the job.
- "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing.
- the test compiler does not appear as a dependency for
the test logs that use it. This is most useful when
RECHECK_LOG =.
- it would be equally useful to be able to add dependencies
to a category of test drivers. Say, with the documented
example:
TESTS = foo.pl bar.py baz
TEST_EXTENSIONS = .pl .py
PL_LOG_COMPILER = $(PERL)
AM_PL_LOG_FLAGS = -w
PY_LOG_COMPILER = $(PYTHON)
AM_PY_LOG_FLAGS = -v
LOG_COMPILER = ./wrapper-script
AM_LOG_FLAGS = -d
be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and
LOG_DEPENDENCIES. I don't think EXTRA dependencies would
be useful.
Cheers!
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#11287: Various issues with the test suite framework |
|
Date: |
Mon, 02 Jul 2012 21:27:22 +0200 |
Reference: <http://debbugs.gnu.org/11287>
To sum up the actions related to this bug report:
On 04/20/2012 01:13 PM, Akim Demaille wrote:
> Hi!
>
> I have seen that check-html will be removed, or rather moved
> into the contrib part, but there are a few issues:
>
> - the target is not declared recursive, so one has to write
> the bouncing target herself.
>
This isn't a big deal anymore, now that the user can define extra
recursive targets through the macro AM_EXTRA_RECURSIVE_TARGETS
(features introduced in commit v1.12.1-163-g3da1457); the test
'contrib/t/parallel-tests-html-recursive.sh' (introduced by commit
v1.12.1-191-g53d4984) should prove this assertion.
> - because of that, the "naive" implementation of check-html
> that just bounces the right directories will miss the
> dependencies that check features (in other words, make
> check properly recursively builds before, while check-html
> would just go into the typical tests/ directory, and then
> bounce to "check", so it completely missed the recursion
> in the other directories and their dependencies).
>
Adding 'check-html' to AM_EXTRA_RECURSIVE_TARGETS should take
care of this issue as well.
> - RECURSIVE_TARGETS cannot be highjacked by the users, that's
> sad. Could a $(USER_RECURSIVE_TARGETS) be added to
> RECURSIVE_TARGETS so that we can add our own targets that
> traverse the whole package? Actually maybe this should
> be an Automake option to add recursive targets, and
> then TARGET-am and TARGET-recursive would be generated and
> bound to the proper TARGET.
>
Addressed by commit v1.12.1-163-g3da1457 (with a different design
but comparable functionalities); see this thread for more info:
<http://lists.gnu.org/archive/html/automake-patches/2012-06/msg00195.html>
> - the .log.html recipe features $$RST2HTML, but in '', so it is
> useless. And anyway $(RST2HTML) already does the job.
>
Agreed. Fixed by commit v1.12.1-192-g18dac90.
> - "$(AM_RST2HTMLFLAGS) $(RST2HTMLFLAGS)" is missing.
>
Fixed by commit v1.12.1-193-g67abae4.
> - the test compiler does not appear as a dependency for
> the test logs that use it.
>
Reporting what I've stated in an earlier reply:
This makes sense, since the test compiler is not required to be
an in-project script, and not even to be specified as an absolute
path -- it can be a command like "sh" or "/usr/bin/env perl".
> (This is most useful when RECHECK_LOG =.)
> - it would be equally useful to be able to add dependencies
> to a category of test drivers. Say, with the documented
> example:
>
> TESTS = foo.pl bar.py baz
> TEST_EXTENSIONS = .pl .py
> PL_LOG_COMPILER = $(PERL)
> AM_PL_LOG_FLAGS = -w
> PY_LOG_COMPILER = $(PYTHON)
> AM_PY_LOG_FLAGS = -v
> LOG_COMPILER = ./wrapper-script
> AM_LOG_FLAGS = -d
>
> be able to add PL_LOG_DEPENDENCIES, PY_LOG_DEPENDENCIESS and
> LOG_DEPENDENCIES. I don't think EXTRA dependencies would
> be useful.
>
This would be tricky to implement in the current Automake code base, and
IMHO for a too little benefit; so I'm going to label this part of the
report as a "wontfix". However, it was easy to implement in Automake-NG:
<http://lists.gnu.org/archive/html/automake-ng/2012-05/msg00022.html>
-*-*-*-
All that considered, I'm closing this bug report now.
Regards,
Stefano
--- End Message ---