[Axiom-developer] Literate Programming and Reproducible Results

[TimDaly]

The Ecole Polytechnique Federale De Lausanne has introduced
an online journal fore reproducible research at:
  <http://rr.epfl.ch/17/>

The introductory headline reads:

Have you ever tried to reproduce the results presented in a research
paper? For many of our current publications, this would unfortunately
be a challenging task. For a computational algorithm, details such as
the exact dataset, initialization or termination procedures and
precise
parameter values are often omitted in the publication for various
reasons. This makes it difficult, if not impossible, for someone else
to
obtain the same results. To address the problem, we have started
making our research reproducible. Instead of only describing the
developed algorithms to ‘sufficient’ precision in an article, we give
readers access to all the information (code, data, schemes, etc.)
that
was used to produce the presented results as first advocated by Knuth
and Claerbout. We are convinced that making research reproducible is
not only a matter of good practice, but also increases the impact of
our
publications and makes it easier to build upon each other’s work. It
is a
clear win-win situation for our community: we will have access to
more
and more algorithms and can spend time inventing new things rather
than recreating existing ones.


I am a firm believer in Knuth's Literate Programming and in the need
to have full publication of the program as well as the research paper
in the field of computational mathematics (CM).

Too often in the CM area the only surviving artifact of an algorithm
is
a 5 page paper in a conference publication. There is no need to limit
publications to the constraint of paper. Conference proceedings can
be published electronically. With the adoption of some reasonable,
minimal standards it should be possible to "drag-and-drop" a paper
onto a running system and have it installed immediately. Indeed,
it could even occur at the conference while attending the talk which
would allow the audience to run the algorithm immediately.

Combining the research paper with the actual code will raise the
standard of publication and encourage standardization of the CM
algorithms. If, in addition, the publication license allowed others
to make derivative algorithms we could see a whole history of the
development of important results (say, new Groebner basis
enhancements) with a clear history of prior art.

Tim Daly