|
From: | Jacob Bachmeyer |
Subject: | bug#42399: DejaGnu randomly truncates lines in runtest output |
Date: | Thu, 16 Jul 2020 21:40:21 -0500 |
User-agent: | Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1.22) Gecko/20090807 MultiZilla/1.8.3.4e SeaMonkey/1.1.17 Mnenhy/0.7.6.0 |
Alex Coplan wrote:
Then, create a subdirectory bug.dg, containing a single file bug.exp with the following contents: global text spawn "./a.out" set prefix "\[^\r\n\]*" expect { -re "^$prefix\tPASSED:${text}*" { regsub "\[\n\r\t\]*PASSED: $text\r\n" $expect_out(0,string) "" output set output [string range $output 8 end] pass "$output" exp_continue } -re "^$prefix\r\n" { exp_continue } }
The bug is actually in bug.exp: you have an expect pattern that does not require a newline at the end of its match. Most of the time, DejaGnu runs often enough that there is only one line in the buffer, so this happens to work, but if the program under test produces output very quickly (as C unit test executables tend to do) the block read into DejaGnu may not end on a line boundary. If the patterns require a terminating newline, there is no issue, since the patterns can then only match whole lines and Expect will read more input if needed.
There are plans to document the DejaGnu native unit testing protocol and to refactor reading test results into a separate API call from running the program under test to provide better support for other unit testing protocols like Perl's TAP, but the host_execute procedure currently exhibits this bug in master.
However, we are close to a 1.6.3 release and changing host_execute could have far-reaching effects. The bad patterns in host_execute go back to the earliest revisions in the repository, and I hesitate to that procedure at this time, so this may have to be a known bug in the 1.6.3 release, in which case it will be added to the planned fixes list for 1.6.4.
To Rob Savoye: should we delay the 1.6.3 release to change the patterns in host_execute? I think that I can adapt the needed patterns from the DejaGnu testsuite.
Running the last step multiple times, you should be able to see that the truncation happens at different points in the output each time. This nondeterministic behaviour makes it especially problematic since it is now quite difficult to compare results across multiple test runs.
Yes, the truncations are a result of the whims of the system scheduler.There are two parts to this issue: (1) the documentation needs to warn of this caveat and explain that patterns must not match prefixes of their intended input, and (2) DejaGnu's own host_execute procedure needs to heed that warning. Any similar code in the GCC testsuite also needs to handle this issue.
-- Jacob
[Prev in Thread] | Current Thread | [Next in Thread] |