[Axiom-developer] GSoC CAS-neutral test suite project

[Ted Kosan]

On Wed, 03 Mar 2010 06:40:49 -0800, Tim Daly
wrote to [Axiom-developer] Re: [sage-devel] Randomised testing against
Mathematica

>There are two test suites with validated results at
>http://axiom-developer.org/axiom-website/CATS/
>
>The CATS (Computer Algebra Test Suite) effort targets
>the development of known-good answers that get run
>against several systems. These "end result" suites test
>large portions of the system. As they are tested against
>published results they can be used by all systems.
>
>The integration suite found several bugs in the published
>results which are noted in the suite. It also found a bug
>introduced by an improper patch to Axiom.
>
>It would be generally useful if Sage developed known-good
>test suites in other areas, say infinite sequences and series.
>Perhaps such a suite would make a good GSOC effort with
>several moderators from different systems.
>
>I have done some more work toward a trigonometric test
>suite. So far I have found that Mathematica and Maxima
>tend to agree on branch cuts and Axiom and Maple tend
>to agree on branch cuts. The choice is arbitrary but
>it affects answers. I am having an internal debate about
>whether to choose MMA/Maxima compatible answers just to
>"regularize" the expected results users will see.
>
>Standardized test suites give our users confidence that
>we are generating known-good results for some (small)
>range of expected inputs.
>
>An academic-based effort (which Axiom is not) could
>approach NIST for funding an effort to develop such
>suites. NIST has a website (http://dlmf.nist.gov/)
>Digital Library of Mathematical Functions. I proposed
>developing Computer Algebra test suites for their
>website but NIST does not fund independent open source
>projects. Sage, however, could probably get continuous
>funding to develop such suites which would benefit all
>of the existing CAS efforts.
>
>NSF might also be convinced since such test suites raise
>the level of expected quality of answers without directly
>competing against commercial efforts. I'd like to see a
>CAS testing research lab that published standardized
>answers to a lot of things we all end up debating, such
>as branch cuts, sqrt-of-squares, foo^0, etc.
>
>Tim Daly

MathPiper (http://mathpiper.org) is the CAS which is used by GeoGebra
(http://geogebra.org) and GeoGebra was recently accepted as a Google
Summer of Code project.  The GeoGebra project has invited the
MathPiper project (which I lead) to participate in their GSoC effort
and I immediately thought that your idea of a CAS-neutral test suite
would be a good project candidate.

Would you be interested in helping to locate and mentor a GSoC student
to work on a CAS-neutral test suite which is based on the ideas you
discussed in the above email?

Ted Kosan