Re: [Bug-dejagnu] BUG: improper format string construction in framework.exp

[Jacob Bachmeyer]

Ben Elliston wrote:
> On Mon, Oct 29, 2018 at 11:17:33PM -0500, Jacob Bachmeyer wrote:
>   
> > Overwriting site.exp appears to actually be important, because the
> > DejaGnu testsuite needs srcdir to be .../dejagnu/testsuite and
> > Automake sets srcdir to .../dejagnu; thus the need for "--srcdir
> > $(srcdir)/testsuite" in RUNTESTDEFAULTFLAGS in Makefile.am.  How
> > best to fix this misintegration with Automake?
> >     
>
> You can tell DejaGnu to use a different site.exp using $DEJAGNU. Why
> not create a different site.exp for the testsuite (testsuite/site.exp)
> and then set $DEJAGNU in the Makefile rule?
>
> Would that work?
>   

Part of the problem is that the site.exp Automake produces actually is 
important -- it provides information about where the sources are when 
building in a separate directory.  Automake is correct here, and the 
DejaGnu testsuite (and possibly DejaGnu itself) is wrong.

The DejaGnu manual states that a testsuite is required to be in a 
directory named "testsuite" that should be in the tool's source 
directory.  Considering DejaGnu's history with Cygnus, I believe that 
Automake's $(srcdir) is the "tool's source directory" mentioned here.

The srcdir for the DejaGnu testsuite is overridden on the command line, 
but gets partially overridden by the Automake-provided site.exp.  
Fortunately, the only effect of this seems to be an inability to find 
tool init files, which will be a problem when the MULTIPASS test expects 
"set MULTIPASS ..." to be executed from its tool init file, but is not a 
problem at the moment, since libdejagnu.exp is empty and runtest.exp 
appears to be unneeded.

While the command line srcdir takes precedence over a value from 
site.exp, there is a window between reading site.exp and reparsing the 
command line, and the search for the tool init file occurs in this window.

I suggest adding a search path for the tool init file and specifically 
supporting running DejaGnu at both top-level and in the testsuite 
directory.  (Both of those can currently work, so breaking either would 
be bad.)  I suggest searching for a tool init file on

 
${srcdir}/testsuite/lib/tool/${tool}.exp:${srcdir}/testsuite/lib/${tool}.exp:${srcdir}/lib/${tool}.exp

The last item on that path is the current behavior, on which some 
testsuites may depend.  The first item is an enhancement proposal to 
allow tool init files to be kept separate from local testsuite library 
modules.  Additional processing during the search would eliminate 
"testsuite/testsuite/" pairs if ${srcdir} ends with "testsuite".

Unless overridden, DejaGnu's srcdir should match Automake's srcdir.

There will need to be a different site.exp for the options tests 
eventually, but that is a different problem from DejaGnu misusing the 
Automake-provided site.exp.

Also, where in the documentation should I insert a section for the 
configuration files DejaGnu searches?  A new section in Reference?

Lastly, is a soft dependency on GNU awk acceptable for DejaGnu or would 
I need to rewrite my testsuite summarization tool in Tcl or Expect 
before contributing it for use with the DejaGnu testsuite?

-- Jacob