[Axiom-developer] A modest proposal

[daly]

William,

Rather than starting by refuting your points in detail let me ask the
question....If you know that literate programming is a PRIMARY design
goal of Axiom and you don't support that goal, then wouldn't it make
sense to take the freely available original sources, which are not
literate, and start a new Sourceforge project (say, OpenAxiom,
RawAxiom, whatever)? That way you can define the goals to be anything
you like.

Axiom, as it exists in this project, is an experiment in developing
software designed to be maintained, modified, and expanded by people
at least 30 years in the future. The fundamental conjecture is that
without documentation complex code cannot survive.

To that end the Axiom project adopted Knuth's technology of literate
programming.  To quote Patrick McPhee:

   Without wanting to be elitist, the thing that will prevent 
   literate programming from becoming a mainstream method is that
   it requires thought and discipline. The mainstream is established
   by people who want fast results while using roughly the same
   methods that everyone else seems to be using, and literate
   programming is never going to have that kind of appeal. This 
   doesn't take away from the usefulness of the approach.




Indeed I hear, underlying email on this list, both kinds of complaint:

  a) "we want fast results" ...therefore throw out the trunk and let the
     latest, greatest, branch win. This is the essence of your argument.
     So what do we do when Mark Botch shows up with the latest, greatest
     branch? Or Stephen's branch suddenly has a new feature we want? Do
     we throw Waldek's out, rinse and repeat? Surely this is the "fastest
     way" to get the new features. Do we each "do our own thing" and
     fail to contribute?

Fast is NOT a project goal. Literate Programming with deep documentation 
is a project goal. Correctness is a project goal. Being a research 
platform for new ideas is a project goal. But FAST is not. There is no
need to "get it running by September". Development that requires thought
and discipline takes time. We have time. There are no deadlines. The
Axiom project has a "30 year horizon".


  b) "using roughly the same methods that everyone else seems to be using"...
     which is exactly how Scratchpad was developed. William Sit came to
     visit at IBM Research, wrote some code, documented nothing and left.
     I don't object to this kind of programming and I am not advocating
     that all projects should switch to literate programming everywhere.
     But the Axiom project on Sourceforge and Savannah has a PRIMARY
     goal of developing a literate CAS using Knuth's technology.




You state:
> To me, it would be a waste of time to document carefully at that 
> stage (...[snip]...). it only need to be for personal use;

That's fine. Do anything you want for personal use. But if you want to
write code in the Axiom project and you want it in the distribution
then your "personal code" in your "own branch" has graduated to the
stage that an unknown number of people will use it, maintain it,
modify it, and support it. So the "Axiom Distribution" strives for
literate documentation, written for people first, not machines.

If you spend the time to figure out how an algorithm works (e.g.
how the compiler resolves types, how hyperdoc builds pages, or
how the databases are structured (see src/interp/daase.lisp)
then we need you to leverage that effort so that we can understand,
maintain, and modify it later. Otherwise it gets lost. It gets 
"too hard to merge". 




You state:
> and during the development cycle I don't want to be distracted by 
> the rigid format required by a pamphlet. And besides, that format
> is still under experimentation

I don't recall that we've defined the shape of pamphlets. Ralf has
done pioneering work with ALLPROSE. I've made some initial attempts
with DH matrices and quaternions. Pamphlets are hardly a rigid format.
They certainly don't constrain your development or experimentation.
They are only latex files with some special tags. And in the near
future they will be pure latex files once we write latex macros
that replace the current chunk-name syntax for the special tags.
So they are not "home-grown" technology but general-purpose technology
over 20 years old with a known-good example (See TeX: The Program by
Knuth ISBN 0-201-13437-3)



You state:
> So, yes, my Axiom code should be better documented, but not in a pamphlet.

Which raises two questions. First, why are you trying to associate yourself
with a project which has the goal of putting everything in a pamphlet? 
Second, you're not doing documentation anyway so why is this an issue?



You state:
> I believe the style of programming (for example, choosing meaningful
> variable identifiers and function interfaces) is far more important
> for clarity of the code than any documentation.

That's a nice belief. And it is not wrong, just trivial. You surely
chose the variable names in your algebra code to be insightful. Or they
would be if we understood the theory behind the code, the mindset that
you had when you wrote it and unit tests to see what it actually does.
I assure you that my variable names chosen in Axiom, most times, have
followed this. I certainly do this every time I program. So tell me,
how does the interpreter resolve types? Reference the variable names,
explain the algorithm by incanting the variable names in some order,
and show your work. You have one hour.




You state:
> Why should I now repackage all these files into one huge pamphlet, 
> broken into "chunks", intersperse explanation of data-structure, and
> code among theory, distracting from the flow of the theoretical
> development.....

Why bother to use chapter 2 of a dissertation to prove a lemma?
Why introduce the theory of networks to explain the FFT algorithm?
Why not give students the final theorems in the course and simply
state "All the results follow from the theorems, which have carefully
chosen names"? 

These pamphlet files are intended to be read by humans, learned by
humans, and maintained by humans. Pamphlet should explain the "why",
the motivation, the theory, the ideas, the magic. None of that is
in the code no matter how clever you are in choosing variable names.




You write:
> although you ignored the phrase "that is not final" in my argument

No, I didn't ignore it. I'm simply asking when you might consider
work final? (Clearly Manual Bronstein's work is final. He's dead.
However he and his wife have given permission for Axiom to use his
work to document Axiom. That's a future task.) You are still around.
When is your work "final" and when do you think you'll document it?
Are we going to pass the documentation task onto the future? Why will
they be any more inclined to document your work than you are? Do you
really want some third-rate pseudo-mathematician like myself doing it?

If the new build system is stable (and this is entirely up to the
discretion of the developer) and it is going to be the basis for the
new build system then it needs to be documented. One of the primary
reasons is that, of all of the people, I use the build system on a
daily basis. For the past few years I've almost always had at least
one system doing a build (albeit with ancient, buggy, worthless Makes
that don't use standard methods). So of all of the people this 
change will impact I am surely center stage.


The Axiom project has no fundamental opinions about which SCM to
use. We argue over SVN, CVS, Git, Arch, etc. All of that can be
changed. Even the build system can be replaced eventually.  But
literate programming is fundamental. The Axiom project, as it is
defined, is literate.


It is NOT important for the goals of this project that we "get there
FAST" or that "we use the standard methods". If those are your needs
then, by all means, start a Sourceforge project that achieves those
goals. Make a Scratchpad or FASTAxiom or STANDARDAxiom or SITAxiom
project. Take a copy of Waldek's branch and release it under that
name.  Axiom, and by that I mean, THIS Sourceforge project, does not
have those goals. My ONLY request is that you choose a different name
for your new project and stop referring to it as "Axiom".



I freely admit that the goals of the project "Axiom" as it exists
today flow from my prior experience with IBM Axiom. When I got the
distribution from NAG and did all of the work to set up and build this
project I hoped that people would be inspired to achieve something new
and different.  Would they build a literate CAS? And a CAS that stresses
correctness, possibly provable correctness. A CAS that can form the
basis of a science of computational mathematics for the next hundred
years. A CAS to carry on the research tradition with fundamental
changes like provisos. Will people be motivated to take the time to
"do it right"? Can people be motivated to work toward a future they
will never see?

If you want to work on this project please respect the fundamental goals.
That's not an unreasonable request.

Tim