bug#11814: The test logs lost their title

[Stefano Lattarini]

> Le 29 juin 2012 à 19:01, Stefano Lattarini a écrit :
> 
>> Hi Akim.
> 
> Hi!
> 
>> On 06/29/2012 09:17 AM, Akim Demaille wrote:
>>> It seems that in recent changes, the test logs have lost
>>> their title, which included the exit status.  Now, reading
>>> a log, one can no longer know how the test exited.
>>>
>> All of this is intended, because we now report "metadata" about the
>> test run in the '*.trs' file rather than in the '*.log' file.
> 
> This is not metadata, this is the result of the test case.
> When reading the log of the test, I expect to get the result.
> Of course I could do it myself, but it means addition complexity
> with temporary files, trap etc. which I am very happy to avoid.
> Was there a _problem_ that needed to be solved by the removal
> of the result?
>
The problem was that displaying the exit code at the beginning of the
log required one more 'cat' invocation and one more 'mv' invocation per
test.  So, in the heavy refactoring taking place during the introduction
of support for TAP and for user-defined test drivers, I zapped that
feature for the sake of performances, aware of the fact that the exit
status was still available in the new corresponding '.trs' file.  But I
see your point about the convenience of having it repeated in the log
file too.  So I will add a new option for the default driver, as well as
for the TAP driver, that will cause them to print a summary of the test
output at the end of the '.log' file.

> 
>> So,
>> to know whether a test run has been successful, or which the results
>> of the individual subtests has been, you should look at that file
>> (it should be easily human readable as well as machine parseable,
>> although its format is still experimental and likely to change in
>> future Automake versions).
> 
> I don't want to read several files.
> 
> Also, test-suite.log contains a new section which
> is not compliant with rst:
> 
>> ==========================================
>>    Vaucanson 2.0a: tests/test-suite.log
>> ==========================================
>>
>> # TOTAL: 16
>> # PASS:  14
>> # SKIP:  0
>> # XFAIL: 1
>> # FAIL:  1
>> # XPASS: 0
>> # ERROR: 0
>>
>> .. contents:: :depth: 2
>>
>> XFAIL: rat/wrong-weight-set
>> ===========================
> 
> which results in:
>
> [SNIP IMAGE]
>
I don't know reStructuredText well enough to fix this without too much
effort.  But of course, I'll gladly accept a patch (better if with a
test case, unless that would be too tricky or too hard to write).

Regards,
  Stefano