Re: [Chicken-janitors] #630: Test scripts that don't properly report exit status

[Chicken Trac]

#630: Test scripts that don't properly report exit status
-------------------------+--------------------------------------------------
  Reporter:  mario       |       Owner:                     
      Type:  defect      |      Status:  new                
  Priority:  major       |   Milestone:  4.8.0              
 Component:  extensions  |     Version:  4.7.x              
Resolution:              |    Keywords:  test infrastructure
-------------------------+--------------------------------------------------
Description changed by iraikov:

Old description:

> The eggs listed bellow don't properly report their exit code to the test
> infrastructure.
>
> Please, make them exit 0 when all tests pass or a non-zero value when one
> or more tests fail, so salmonella can properly indicate the tests status.
> Hint: for those using the [http://wiki.call-cc.org/egg/test test egg],
> you just need {{{(test-exit)}}} at the end of the test script.
>
> If you test script relies on errors (or {{{assert}}}) to test egg
> features, you can assume it is ok (I'm sorry for the false alert), since
> on error the test script will exit 70 (which is ok for the test
> infrastructure).
>
> I took a look at the last 20 lines of the run.scm files (or at the files
> they include/load) in the trunk dir for each egg (yeah, it's not exactly
> a pleasant work).  If you don't intend to make a new release of your
> egg(s) to fix bugs or to add new features soon, please consider making a
> new release just to fix the tests, so we can have a more accurate idea
> about the shape of the set of eggs.  Also consider that egg tests help to
> test the compiler too.
>

> atlas-lapack
> awk
> big-chicken
> binary-heap
> bind
> bindings
> blas
> blob-record
> byte-blob
> byte-blob-stream
> char-set-literals
> cis
> condition-utils
> contracts
> coops-utils
> couchdb
> csv
> datatype
> digraph
> directory-utils
> discount
> dollar
> dsssl-utils
> elliptic-curves
> endian-blob
> err5rs-arithmetic
> expand-full
> fast-generic
> filepath
> fmt
> format-graph
> format-textdiff
> format
> fps
> getopt-long
> glpk
> graph-bfs
> graph-cycles
> graph-dfs
> graph-scc
> heap-o-rama
> html-form
> htmlprag
> iexpr
> inline
> internet-message
> internet-timestamp
> interval-digraph
> isbn
> json-abnf
> kanren
> lazy-ffi
> lexgen
> list-utils
> logpro
> lookup-table
> loop
> macosx
> mailbox-threads
> mailbox
> mathh
> memcached
> mime
> modular-arithmetic
> mpi
> mw-core
> mw
> netstring
> numbers
> numspell
> object-graph
> openal
> operations
> packedobjects
> pandora
> procedure-decoration
> progress-indicators
> prometheus
> protobj
> quaternoins
> rb-tree
> remote-mailbox
> ripemd
> s11n
> sassy
> sexp-diff
> sfht
> sha2
> smtp
> snowdigest
> sparse-vectors
> specialized-io
> spock
> srfi-101
> srfi-19
> srfi-25
> srfi-27
> srfi-42
> srfi-45
> stack
> stalin
> statistics
> string-utils
> sxml-templates
> symbol-utils
> synch
> system
> tigher-hash
> tinyclos
> tiny-prolog
> treap
> tuples
> typeclass
> uuid-ossp
> xosd
> yasos
> yelp
> z3

New description:

 The eggs listed bellow don't properly report their exit code to the test
 infrastructure.

 Please, make them exit 0 when all tests pass or a non-zero value when one
 or more tests fail, so salmonella can properly indicate the tests status.
 Hint: for those using the [http://wiki.call-cc.org/egg/test test egg], you
 just need {{{(test-exit)}}} at the end of the test script.

 If you test script relies on errors (or {{{assert}}}) to test egg
 features, you can assume it is ok (I'm sorry for the false alert), since
 on error the test script will exit 70 (which is ok for the test
 infrastructure).

 I took a look at the last 20 lines of the run.scm files (or at the files
 they include/load) in the trunk dir for each egg (yeah, it's not exactly a
 pleasant work).  If you don't intend to make a new release of your egg(s)
 to fix bugs or to add new features soon, please consider making a new
 release just to fix the tests, so we can have a more accurate idea about
 the shape of the set of eggs.  Also consider that egg tests help to test
 the compiler too.


 awk
 big-chicken
 bind
 bindings
 char-set-literals
 condition-utils
 contracts
 coops-utils
 couchdb
 csv
 datatype
 digraph
 directory-utils
 discount
 dollar
 dsssl-utils
 elliptic-curves
 endian-blob
 err5rs-arithmetic
 expand-full
 fast-generic
 filepath
 fmt
 format-graph
 format-textdiff
 format
 fps
 getopt-long
 glpk
 graph-bfs
 graph-cycles
 graph-dfs
 graph-scc
 heap-o-rama
 html-form
 htmlprag
 iexpr
 inline
 internet-message
 internet-timestamp
 interval-digraph
 isbn
 json-abnf
 kanren
 lazy-ffi
 lexgen
 list-utils
 logpro
 lookup-table
 loop
 macosx
 mailbox-threads
 mailbox
 mathh
 memcached
 mime
 modular-arithmetic
 mpi
 mw-core
 mw
 netstring
 numbers
 numspell
 object-graph
 openal
 operations
 packedobjects
 pandora
 procedure-decoration
 progress-indicators
 prometheus
 protobj
 quaternoins
 rb-tree
 remote-mailbox
 ripemd
 s11n
 sassy
 sexp-diff
 sfht
 sha2
 smtp
 snowdigest
 sparse-vectors
 specialized-io
 spock
 srfi-101
 srfi-19
 srfi-25
 srfi-27
 srfi-42
 srfi-45
 stack
 stalin
 statistics
 string-utils
 sxml-templates
 symbol-utils
 synch
 system
 tigher-hash
 tinyclos
 tiny-prolog
 treap
 tuples
 typeclass
 uuid-ossp
 xosd
 yasos
 yelp
 z3

--

-- 
Ticket URL: <https://bugs.call-cc.org/ticket/630#comment:3>
Chicken Scheme <http://www.call-with-current-continuation.org/>
Chicken Scheme is a compiler for the Scheme programming language.