[Bug-dejagnu] Hello world example won't report failure (rereresend, edited)

[Flash Sheridan]

Summary:  DejaGnu's "Hello World" example (#10 in "DejaGnu: The GNU  
Testing Framework") will not report a failure if one occurs.  Adding  
"default {fail $test}" fixes this.


Setup:

Macintosh OSX:
Expect version is       5.42.0
Tcl version is          8.4
Framework version is    1.4.4

Ubuntu Linux 5.04:
Expect version is       5.42.1
Tcl version is          8.4
Framework version is    1.4.4

I haven't installed the preceding calc example; in my opinion, "Hello  
World" should come first.


Steps to Reproduce:

I copied the file local_echo.exp from Example 10 in the  
documentation, but with the expected pattern altered to force a failure:

> 34> cat testsuite/hello.test/local_echo_no.exp
> set test "Local Hello World"
> send "echo Hello World"
> expect {
> -re "Hello_NOT World" { pass "$test" }
> }

Running this, and entering no text, does _not_ report a failure:

> 25> runtest   testsuite/hello.test/local_echo_no.exp
> WARNING: No tool specified
> Test Run By flash on Wed Feb  1 13:23:06 2006
> Native configuration is i686-pc-linux-gnu
>
>                 ===  tests ===
>
> Schedule of variations:
>     unix
>
> Running target unix
> Using /usr/share/dejagnu/baseboards/unix.exp as board description  
> file for target.
> Using /usr/share/dejagnu/config/unix.exp as generic interface file  
> for target.
> WARNING: Couldn't find tool config file for unix, using default.
> Running ./testsuite/hello.test/local_echo_no.exp ...
> echo Hello World
>                 ===  Summary ===

Note the absence of an "unexpected failures" line.

Adding "default {fail $test}" makes the test correctly report a  
failure.  In the following session I entered no text; similar results  
when entering "Hello World".

> 39> cat  testsuite/hello.test/local_echo_no_fixed2.exp
> set test "Local Hello World"
> send "echo Hello World"
> expect {
>         -re "Hello_NOT World" { pass "$test" }
>         default {fail $test}
> }

> 37> runtest   testsuite/hello.test/local_echo_no_fixed2.exp  
> WARNING: No tool specified
> Test Run By flash on Wed Feb  1 15:53:06 2006
> Native configuration is i686-pc-linux-gnu
>
>                 ===  tests ===
>
> Schedule of variations:
>     unix
>
> Running target unix
> Using /usr/share/dejagnu/baseboards/unix.exp as board description  
> file for target.
> Using /usr/share/dejagnu/config/unix.exp as generic interface file  
> for target.
> WARNING: Couldn't find tool config file for unix, using default.
> Running ./testsuite/hello.test/local_echo_no_fixed2.exp ...
> echo Hello WorldFAIL: Local Hello World
>
>                 ===  Summary ===
>
> # of unexpected failures        1


The calc.exp example has an expect_after which looks like it should  
have a similar effect, but that seems too complicated for a novice.
I'm not sure whether (a) the author intended this case to work  
without the manual entry of text, or (b) the emitted prompt is  
misleading, but either way, that's a lesser issue than undetected  
failure.

---
http://pobox.com/~flash
Quality Lead for Compilers and Debuggers
PalmSource, Inc. Tools Quality Management