[Automake-commit] [SCM] GNU Automake branch, GSoC/experimental/test-results-work, created. v1.11-901-g17b1fce

[Stefano Lattarini]

This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "GNU Automake".

http://git.sv.gnu.org/gitweb/?p=automake.git;a=commitdiff;h=17b1fce52099a76111920ed209157880605fefae

The branch, GSoC/experimental/test-results-work has been created
        at  17b1fce52099a76111920ed209157880605fefae (commit)

- Log -----------------------------------------------------------------
commit 17b1fce52099a76111920ed209157880605fefae
Author: Stefano Lattarini <address@hidden>
Date:   Sun Jul 3 19:20:22 2011 +0200

    tap: add experimental TAP-aware driver

commit 8714cd90c2e5e074eb773a94f5884eed196cfa95
Author: Stefano Lattarini <address@hidden>
Date:   Mon Jul 4 18:12:07 2011 +0200

    tests defs: new auxiliary function 'count_test_results'
    
    * tests/defs (count_test_results): New function.
    * tests/check11.test: Use it.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-custom-multitest-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise, and
    slightly improve debugging output.
    * tests/parallel-tests.test: Make use of `count_test_results'.
    Also, make grepping of "make check" output slightly stricter
    * tests/parallel-tests9.test: Likewise.
    * tests/parallel-tests-log-override-2.test: Likewise, and throw
    in a small optimization.

commit f9ee7ecdc14ff5eb2917406d13a008a7f8ea5ae4
Author: Stefano Lattarini <address@hidden>
Date:   Fri Jul 1 11:12:31 2011 +0200

    parallel-tests: simplify testsuite summary
    
    Prefer a more deterministic, "tabular" format for the testsuite
    summary, always listing the numbers of passed, failed, xfailed,
    xpassed, skipped and errored tests, even when these numbers are
    zero.  This simplify the logic of testsuite summary creation,
    makes it more easily machine-parseable, and will probably allow
    for easier addition of new kinds of test results in the future.
    
    * lib/am/check.am (am__tty_colors_dummy): New make variable, to
    reduce code duplication.  Extracted from previous versions of
    $(am__tty_colors), and extended by defining two new variables
    `$mgn' and `$brg'.
    [%?COLOR%, %!?COLOR%] (am__tty_colors): Use that new variable.
    (am__text_box): Delete, is not needed anymore.
    ($(TEST_SUITE_LOG)): Rewrite associated rules to implement the
    new testsuite summary format.
    * NEWS: Update.
    * tests/check10.test: Don't run with the parallel-tests harness
    too, that makes no sense anymore.
    * tests/color.test: Update and adjust.
    * tests/color2.test: Likewise.
    * tests/parallel-tests.test: Likewise.
    * tests/parallel-tests3.test: Likewise.
    * tests/parallel-tests6.test: Likewise.
    * tests/parallel-tests9.test: Likewise.
    * tests/parallel-tests-unreadable-log.test: Likewise.
    * tests/parallel-tests-empty-testlogs.test: Likewise.
    * tests/parallel-tests-log-override-recheck.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/parallel-tests-no-color-in-log.test: New test.
    * tests/testsuite-summary-color.test: Likewise.
    * tests/testsuite-summary-count.test: Likewise.
    * tests/testsuite-summary-count-many.test: Likewise.
    * tests/testsuite-summary-reference-log.test: Likewise.
    * tests/testsuite-summary-checks.sh: New auxiliary script, used
    by the new tests above.
    * tests/extract-testsuite-summary: Likewise.
    * tests/trivial-test-driver: Optimize for speed when there are
    lots of of tests.
    * tests/Makefile.am (EXTRA_DIST): Distribute them.
    (testsuite-summary-color.log, testsuite-summary-count.log): Depend
    on them.
    (testsuite-summary-count-many.log): Depend on the auxiliary scripts
    'trivial-test-driver' and 'extract-testsuite-summary'.
    (TESTS): Update.

commit 50e66e604911b170f8435c5585150e6307f3caf4
Author: Stefano Lattarini <address@hidden>
Date:   Thu Jun 30 14:55:43 2011 +0200

    parallel-tests: new recognized test result 'ERROR'
    
    * lib/am/check.am ($(TEST_SUITE_LOG)): Recognize a new test result
    `ERROR'.  Use it when encountering unreadable test logs (previously
    a simple `FAIL' was used in this situations).
    * lib/test-driver: Set the global test result to `ERROR' when the
    test exit status is 99.  When doing colorized output, color `ERROR'
    results in magenta.
    * doc/automake.texi (Log files generation and test results
    recording): Update by also listing `ERROR' among the list of valid
    `:test-results:' arguments.
    * NEWS: Update.
    * tests/trivial-test-driver: Update.
    * tests/parallel-tests.test: Likewise.
    * tests/parallel-tests-harderror.test: Likewise.
    * tests/parallel-tests-no-spurious-summary.test: Likewise.
    * tests/test-driver-global-log.test: Likewise.
    * tests/test-driver-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck.test: Likewise.
    * tests/test-driver-custom-multitest-recheck2.test: Likewise.
    * tests/test-driver-custom-multitest.test: Likewise.
    * tests/test-driver-custom-no-html.test: Likewise.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/color.test: Likewise.  Also, make stricter, and also test
    from VPATH.
    * tests/color2.test: Likewise, and improve syncing with color.test.
    * tests/parallel-tests-exit-statuses.test: New test.
    * tests/parallel-tests-console-output.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

commit 61b2bb0d647ea5f192e49f5f5885009ea83968be
Author: Stefano Lattarini <address@hidden>
Date:   Wed Jun 29 16:02:52 2011 +0200

    parallel-tests: make parsing of test results safer
    
    The new code for parsing the testsuite-generated `.log' files,
    as introduced in commit `v1.11-872-gc96b881', considers each
    `:test-result:' field anywhere in a `.log' file as a declaration
    of a test result, and accounts for it as such in the testsuite
    summary.  Unfortunately this could easily cause spurious test
    failures being reported in the testsuite summary.  This happened
    in practice with the Automake's own testsuite; for example:
    
      $ make check TESTS='check12-p.test'; echo exit: $?
      ...
      PASS: check12-p.test
      =====================================
      4 of 5 tests failed
      See tests/test-suite.log
      Please report to address@hidden
      =====================================
      make[2]: *** [test-suite.log] Error 1
      make: *** [check-am] Error 2
      exit: 2
    
    This change introduces a new special `:test-result:' "END", that,
    when seen, prevents the rest of the log file from being parsed.
    
    For more information, refer to the thread:
    <http://lists.gnu.org/archive/html/automake-patches/2011-06/msg00199.html>
    
    * lib/am/check.am ($(TEST_SUITE_LOG)): Stop the parsing of a log
    file as soon as the special ":test-result:END" directive is seen.
    Related changes and enhancements.
    * lib/test-driver: Protect the rest of the log after the result
    lined with a ":test-result:END" directive.
    * tests/parallel-tests-no-spurious-summary.test: New test.
    * tests/test-driver-end-test-results.test: Likewise.
    * tests/Makefile.am (TESTS): Update.

-----------------------------------------------------------------------


hooks/post-receive
-- 
GNU Automake