[Automake-commit] [SCM] GNU Automake branch, GSoC/experimental/test-results-work, created. v1.11-901-g08cc32c

[Stefano Lattarini]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".

http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=08cc32c938864b29fbf30040ac180c6f4c06dba4

The branch, GSoC/experimental/test-results-work has been created
        at  08cc32c938864b29fbf30040ac180c6f4c06dba4 (commit)

- Log -----------------------------------------------------------------
commit 08cc32c938864b29fbf30040ac180c6f4c06dba4
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 3 19:20:22 2011 +0200

    tap: add experimental TAP-aware driver

commit 863e1bb7a3928a5a885fde5285256a2c0e841671
Author: Stefano Lattarini <address@hidden>
Date:   Mon Jul 4 18:12:07 2011 +0200

    tests defs: new auxiliary function 'count_test_results'
    
    * tests/defs (count_test_results): New function.
    * tests/check11.test: Use it.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-custom-multitest-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise, and slightly
    improve debugging output.
    * tests/parallel-tests.test: Make use of `count_test_results'.
    Also, make grepping of "make check" output slightly stricter
    * tests/parallel-tests9.test: Likewise.
    * tests/parallel-tests-log-override-2.test: Likewise, and throw
    in a small optimization.

commit a452227be3b0f96a2823c5cc1f3f0887f3931090
Author: Stefano Lattarini <address@hidden>
Date:   Fri Jul 1 11:12:31 2011 +0200

    parallel-tests: simplify testsuite summary
    
    Prefer a more deterministic, "tabular" format for the testsuite
    summary, always listing the numbers of passed, failed, xfailed,
    xpassed, skipped and errored tests, even when these numbers are
    zero.  This simplify the logic of testsuite summary creation,
    makes it more easily machine-parseable, and will probably allow
    for easier addition of new kinds of test results in the future.
    
    * lib/am/check.am (am__tty_colors_dummy): New make variable, to
    reduce code duplication.  Extracted from previous versions of
    $(am__tty_colors), and extended by defining two new variables
    `$mgn' and `$brg'.
    [%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable.
    (am__text_box): Delete, is not needed anymore.
    ($(TEST_SUITE_LOG)): Rewrite associated rules to implement the
    new testsuite summary format.
    * NEWS: Update.
    * tests/check10.test: Don't run with the parallel-tests harness
    too, that makes no sense anymore.
    * tests/color.test: Update and adjust.
    * tests/color2.test: Likewise.
    * tests/parallel-tests.test: Likewise.
    * tests/parallel-tests6.test: Likewise.
    * tests/parallel-tests9.test: Likewise.
    * tests/parallel-tests-unreadable-log.test: Likewise.
    * tests/parallel-tests-empty-testlogs.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/parallel-tests-no-color-in-log.test: New test.
    * tests/testsuite-summary-color.test: Likewise.
    * tests/testsuite-summary-count.test: Likewise.
    * tests/testsuite-summary-count-many.test: Likewise.
    * tests/testsuite-summary-reference-log.test: Likewise.
    * tests/testsuite-summary-checks.sh: New auxiliary script, used
    by the new tests above.
    * tests/extract-testsuite-summary: Likewise.
    * tests/trivial-test-driver: Optimize for speed when there are
    lots of of tests.
    * tests/Makefile.am (EXTRA_DIST): Distribute them.
    (testsuite-summary-color.log, testsuite-summary-count.log): Depend
    on them.
    (testsuite-summary-count-many.log): Depend on the auxiliary scripts
    'trivial-test-driver' and 'extract-testsuite-summary'.
    (TESTS): Update.

commit 2e829d8fb966408a0d27409644f2168535c6303c
Author: Stefano Lattarini <address@hidden>
Date:   Thu Jun 30 14:55:43 2011 +0200

    parallel-tests: new recognized test result 'ERROR'
    
    * lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
    `ERROR'.  Use it when encountering unreadable test logs (previously
    a simple `FAIL' was used in this situations).
    * lib/test-driver: Set the global test result to `ERROR' when the
    test exit status is 99.  When doing colorized output, color `ERROR'
    results in magenta.
    * doc/automake.texi (Log files generation and test results
    recording): Update by also listing `ERROR' among the list of valid
    `:test-results:' arguments.
    * NEWS: Update.
    * tests/trivial-test-driver: Update.
    * tests/parallel-tests.test: Likewise.
    * tests/parallel-tests-harderror.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/test-driver-global-log.test: Likewise.
    * tests/test-driver-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-custom-no-html.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/color.test: Likewise.  Also, make stricter, and also test
    from VPATH.
    * tests/color2.test: Likewise, and improve syncing with color.test.
    * tests/parallel-tests-exit-statuses.test: New test.
    * tests/parallel-tests-console-output.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

commit 310e80b275ed120af4999e0dad018b8a5732fd02
Author: Stefano Lattarini <address@hidden>
Date:   Wed Jun 29 16:02:52 2011 +0200

    parallel-tests: make parsing of test results safer
    
    The new code for parsing the testsuite-generated `.log' files,
    as introduced in commit `v1.11-872-gc96b881', considers each
    `:test-result:' field anywhere in a `.log' file as a declaration
    of a test result, and accounts for it as such in the testsuite
    summary.  Unfortunately this could easily cause spurious test
    failures being reported in the testsuite summary.  This happened
    in practice with the Automake's own testsuite; for example:
    
      $ make check TESTS='check12-p.test'; echo exit: $?
      ...
      PASS: check12-p.test
      =====================================
      4 of 5 tests failed
      See tests/test-suite.log
      Please report to address@hidden
      =====================================
      make[2]: *** [test-suite.log] Error 1
      make: *** [check-am] Error 2
      exit: 2
    
    This change introduces a new special `:test-result:' "END", that,
    when seen, prevents the rest of the log file from being parsed.
    
    For more information, refer to the thread:
    <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>
    
    * lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log
    file as soon as the special ":test-result:END" directive is seen.
    Related changes and enhancements.
    * lib/test-driver: Protect the rest of the log after the result
    lined with a ":test-result:END" directive.
    * tests/parallel-tests-no-spurious-summary.test: New test.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

-----------------------------------------------------------------------


hooks/post-receive
-- 
GNU Automake