bug#67312: dejagnu creates testrun.{sum,log}

[Jacob Bachmeyer]

Tom Tromey wrote:
> I've noticed that if I run 'runtest' in the wrong directory -- for
> example in the source directory rather than the appropriate build
> directory -- dejagnu will create files testrun.log and testrun.sum.
>
> In this case these files are not actually useful.
>
> And, runtest already emits some warnings about this:
>
> prentzel. runtest 
> WARNING: testsuite is not in a testsuite/ directory.
> WARNING: Couldn't find the global config file.
> WARNING: No tool specified
>
> I think it would be nicer if, when dejagnu notices that the environment
> is unsuitable (like, no tool specified), then it should just print
> errors and exit -- without creating these files.

Each of the warnings mentioned is a warning and not an error because, 
while those indicate that something is probably wrong, there may be 
testsuites out there using them.  The lack of a testsuite/ directory 
violates a long-documented requirement, but 1.6.3 is the first release 
that actually checks that in order to resolve an ambiguity between 
recursive and non-recursive Automake-generated Makefiles.  (Several 
DejaGnu tests in the Automake testsuite had to be patched to fix exactly 
the misuse that triggers that warning, so there may be other testsuites 
out there that do not properly use testsuite/.  The ambiguity concerns 
whether srcdir is "testsuite" itself or its parent.)  The global config 
file is part of site configuration, is expected to define remote 
targets, and is actually optional if built-in defaults are sufficient, 
as they usually are for the "unix" target.  Lastly, while failure to 
specify a tool produces a warning, it is a valid invocation of runtest.

The problem with bailing out in this case is that failing to specify a 
tool is a valid (legacy, possibly deprecated) use:  it means to run all 
tests (for all tools) in the testsuite.  Writing log and summary files 
as if a tool named "testrun" was tested is the long-standing behavior in 
this case and there is actually explicit code in lib/framework.exp to 
support this.  (That code may also mean that a tool literally named 
"testrun" cannot actually be tested...)

Of the three warnings, the lack of a testsuite/ directory is the best 
candidate to upgrade to a hard error since it has seemingly always been 
a documented requirement, but all previous releases would silently 
accept a testsuite not using that directory, so moving directly to 
"ERROR:  no testsuite/ directory found.  Stop." is too much to do on a 
point release.

Are the produced files empty?  I will consider if there might be a good 
way to detect a completely bogus run (example:  no test scripts found at 
all) and bail out early without creating logfiles, but this issue seems 
to be a valid use case that is misfiring in your situation.


-- Jacob