axiom-developer
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Axiom-developer] regression testing, macro table


From: root
Subject: [Axiom-developer] regression testing, macro table
Date: Fri, 24 Oct 2003 07:28:38 -0400

*,

I previously broke the util.ht file. Somehow I clipped out the
stanza that walks into that subdirectory. That's been fixed.

I put up an early-release version of regression testing.
Basically it works as follows:

* the final build completes
* the input files are generated
* a Makefile is generated in int/input
* the Makefile is invoked which
   - runs the input files
   - saves the output

If you look at the Makefile (the source is in src/input/Makefile.pamphlet
under the <<regression tests>> chunk you can see that I've classified the
working/failing input files by type of failure. That way we have a larger
set of test cases for chasing a particular failure type.

"Failing" means that it causes a lisp break. "Working" means it doesn't.

Note that some of the "working" input files are still buggy for 2 reasons.
First, the names of the types or operators have changed so the input file
function fail to execute. Second, the input fails because it is supposed
to fail. That is, the input file is demonstrating a bug or intentionally
incorrect code. Failures can be found in the output by searching for
"Daly bug" messages The name is a holdover from scratchpad and was in
the NAG version. It is enabled by )set message test on. It exists so
I can automate detecting failures.

The regression testing is cheesy and limp but it is a start.
The Computer Algebra Test Suite (CATS) effort will eventually 
replace it.

Tim
address@hidden
address@hidden





reply via email to

[Prev in Thread] Current Thread [Next in Thread]