bug#41824: Dejagnu's unknown proc aborts testsuite run when triggered in test-case

[Jacob Bachmeyer]

Pedro Alves wrote:
> On 6/30/20 5:40 AM, Jacob Bachmeyer wrote:
>   
> > Pedro Alves wrote:
> >     
> > > On 6/26/20 11:37 PM, Jacob Bachmeyer wrote:
> > >  
> > >       
> > > > Pedro Alves wrote:
> > > >    
> > > >         
> > > > > And again, people run the GDB testsuite in parallel mode, so your
> > > > > assumption that the testsuite would stop immediately anyway doesn't
> > > > > hold.  One of the parallel runtest invocations bombs
> > > > > out, while the others keep running, so by the time you get
> > > > > back the prompt, other testcases already executed, and you don't
> > > > > see the ERROR in the current terminal anymore.
> > > > >       
> > > > >           
> > > > This is a fundamental problem with slicing up the testsuite external to DejaGnu and 
> > > > running multiple instances of runtest -- output that is supposed to be "last at the 
> > > > end of the run" can be hidden away by other ongoing subset runs.
> > > >     
> > > >         
> > > That's like arguing against parallelization, and saying that everyone
> > > should run the testsuite in serial mode, and be OK with it taking 2 hours
> > > instead of 30 mins to complete a run...
> > >
> > > The parallel mode aggregates the summaries from the different runtest
> > > invocations as a single summary at the end,
> > >
> > > https://sourceware.org/git/?p=binutils-gdb.git;a=blob;f=contrib/dg-extract-results.py;h=30aa68771d491451f72d6dbd18f6d5ac339451b5;hb=HEAD
> > >
> > > so I don't see what problem here is unsolvable.  Again, the issue is that Tcl
> > > ERRORs aren't tallied up on the DejaGnu summary, so the aggregated summary
> > > doesn't contain those either.
> > >   
> > >       
> > The current version of the fix for this issue (commit 61dc0cafad8845b3c668940ed2e574bd503d410f on 
> > temporary branch PR41918) generates separator lines containing at least 10 hyphens around each 
> > error when repeating the errors after the summary.  The leading separator line currently begins 
> > with "ERROR: " due to limitations of the internal interfaces, but plans are to eventually 
> > fix that and collapse the separator lines between adjacent error reports if multiple Tcl errors 
> > occur in a run.  Each error report begins with "ERROR: in testcase " and ends with a Tcl 
> > backtrace and a line of at least 10 hyphens.  Is this something the GDB results aggregation tool 
> > can be extended to handle?
> >     
>
> Your guess is as good as mine, since I never touched that code.
>   

After briefly looking through that script, I expect that it will not be 
particularly difficult, although I also note that the script counts the 
error messages generated by DejaGnu's ::unknown procedure, so 
lib/framework.exp:unknown cannot be withdrawn as useless until 1.7.


-- Jacob