Re: [Axiom-developer] Must hear....

[daly]

>> Unless there is some overarching reason it is hard to see the need to
>> rewrite existing code. A new design that regularizes many domains
>> might make a good reason. A complete extension of the whole area, such
>> as a major matrix package might make DHMATRIX a useless subset. But
>> changing the API of an existing domain so that old code doesn't work
>> is, by definition, broken.
>>
>> Mistakes in the code will occur both in old code and in new code.
>> Unfortunately new mistakes are only likely to be uncovered by new
>> people using the new code... which by recursion on the motivation
>> to rewrite the code...  leads to yet a different set of mistakes.
>>
>> Code doesn't rust. It doesn't "get old". Especially computational
>> mathematics code. Tearing down the cathedral because we now know
>> how to make stones that lasts longer seems ... I don't know ...
>> disrespectful of the genius of the original architects?

Perhaps my cathedral analogy was "over the top". That's partly because
I worked with people like Barry Trager, Manuel Bronstein, et.al. The
day-to-day contact made me realize how little I actually understand.
Naturally I am very careful when changing things, trying to understand
what exists before making changes. Google wouldn't let me change
their search algorithm without understanding it and the mathematics
of Axiom is much more complicated.




>One of the ideas I seem to be getting from the Robert Lefkowitz talks is 
>that most of the life-cycle of a piece of software is in maintenance 
>(i.e. change) and that is why this type of documentation is needed. It 
>seemed to me that a corollary of that is that, if the code does not need 
>to change, then documentation is not required.

Axiom is clearly in need of maintenance in so many areas. At the very
minimum we have spent a good portion of the time dealing with low-level
issues like changing libraries, changing tools, and changing operating
systems. Fortunately we are nearing a key goal of a lisp-only platform
so Axiom will only need a working standards-based common lisp. The
build machinery is about to be swallowed and then the front end
hyperdoc work (already started) will be subsumed.

Climbing beyond that level is the effort to document the huge library
of computational mathematics. The code needs to change to handle 
the rest of the Risch algorithm, for instance. But what exists also
needs to be brought to a state where we know what is there, where it
is, and how to extend the missing parts. Manuel gave me permission to
use all of his writings to document the code... It just takes time.

If Axiom was a 100 line program this would be trivial. At 1.2 million
lines it is going to take a while.




>The thought also occurred to me that the wider axiom community has two 
>types of factions, those who want to change without documentation and 
>those who want to document without change. I don't mean this to be taken 
>seriously, its totally unfair of me to write down such an untrue 
>thought. I apologise unreservedly to people who are working very hard to 
>improve a program they believe in strongly.

Well, that's not a bad way to characterize it. I'm re-working the
Axiom emails and I see many cases of changes proposed without even a
single line of comment. I also see a lot of pushback about the wisdom
of LP. So there is certainly a camp of "change without documentation". 
There hasn't been a single, well documented pamphlet file submitted. 
We're also watching LP die in the forks, leading to the "raw code" 
approach.

On the other hand, "documentation without change" seems to
characterize my position on things, mildly unfairly I think, but not
without merit.  I have added algebra for predicting sequences, for
JET, for BLAS, for numeric functions, etc. The numeric code I wrote is
reasonably well documented. The JET code was done as well as I could
from available words. I'd also like to pick up Waldek's recent
integration work but I have no idea where to start, nor what references
to read to understand it, nor enough background to document it. So 
Axiom is changing, slowly.

I've asked several authors for permission to quote their papers, which
is an exception permitted by copyright for research purposes. All but
one have said yes. I have a whole directory of papers slated to be
added as documentation to the related domains. Each one is "expensive"
because I have to learn the relevant background to write readable
documentation and connect it to the domains but, hey, it's a 30 year
horizon project :-)




>I agree about the genius of the original architects who were years ahead 
>of their time. I think more history should be made available to let more 
>people know about this.

>However I don't want to use, or work on, software that is a museum or 
>shrine to the genius of the original architects. I have convinced myself 
>that the sort of changes, that I would like, are driven by real needs to 
>support new mathematical structures and so on and not just a wish to 
>tweak the margins of the program for the sake of it.

There seem to be two philosophical approaches to computational
mathematics. From one side there are the "programmers doing mathematics". 
They follow the usual path of "write the code, read the code". They 
are very liberal about the programs, moving from patch-to-patch, 
changing things. They are fast and liberal, moving at the pace of
programmers everywhere.

On the other side I see the "mathematicians doing programming". They
follow the usual "write the paper, bury the code". They are very 
conservative about the math, moving theorem-to-theorem, proving each
step along the way. Change is slow and conservative, moving at the
pace of mathematicians everywhere. 

As a "computational mathematician" I'm trying to occupy the middle
ground where code gets written and it is intimately connected with the
paper and proof. There is no need to invent new mathematics as there
are whole landscapes of existing work that can be reduced to
programs. The ideas should stay with the code so future Axiom
developers can maintain and modify the mathematics in a disciplined
way. Indeed, a stated long term Axiom goal is to integrate with either
ACL2 or COQ to prove the algorithms in Axiom.  I believe that properly
written literate programs are the best vehicle for all that.

I have published an invited editorial in the Notices of the American
Mathematical Society, pushing for LP and Reproducible Research so
Axiom isn't the only place you'll see me moaning about this. I am
working with John Kitchin from Carnegie Mellon on an effort to teach
the next generation of students to write literate programs. The
hope is that the next generation of students will naturally document
their programs. John works in computational chemistry, which would
make an interesting extension area of Axiom (if I only had time...)





>see my wish list:
>http://www.euclideanspace.com/prog/scratchpad/fricas/wishlist/

I have research in Indefinites, Interval Arithmetic, and Provisos that
I really, really want to implement. I have done work in all three
areas. They would really extend Axiom's power.  I want to do work in
Quantum Physics, with things like Hadamard gates, so I can implement
the quantum fourier transform algorithm. I've been working with Albert
Rich on the computer algebra test suite (CATS) using his 50,000 
integrals. I want to implement his techniques in Axiom to cover the
cases we miss now.

Wish lists are long, days are short.

Axiom is a "spare time" effort, not a career, so time is limited.  I
miss the days when computational mathematics was supported financially.
This "nights and weekends" approach is painfully slow. In fact, without
the support of Gilbert Baumslag, Axiom's release might not have happened
at all.







>I think what you are hinting at is that the original scratchpad software 
>was written by a large number of brilliant people and I am a single, 
>humble individual who is very far from being a genius. I assure you, I 
>know that already. I really should set myself more realistic goals.

There are truly brilliant people I've seen in the open source version
of Axiom. There is no shortage of brilliant people here. Waldek is
doing amazing work, for instance.  Do not ever feel that I'm "hinting"
you're not. I'm not the one to judge, nor is my opinion worth the time
to consider. If you're doing computational mathematics at all, you're
already in the best of company.

We need the brilliant among us to write stuff down for lesser mortals
like myself. It is wonderful to bring the gift of fire ... but could
you explain that trick with rubbing the sticks again? :-)





>It seems to me what Robert Lefkowitz is saying is that programs need to 
>change over time, and for that they need documentation and of course the 
>meaning of the word 'documentation' changes over time and everyone 
>understands the meaning of words differently. Mathematical truth may not 
>change over time, but the way that people use it, what is considered 
>important, notation and just about everything else does change. Perhaps 
>'documentation' does not mean what I think it means but, for me, its 
>about not freezing the program in time.

Mathematics changes over time but very slowly. I've heavily quoted
from a 19th century treatise on quaternions in the Axiom documentation.
The words are still relevant.

I don't want Axiom frozen. I want it brought to a state where it can
be maintained, modified, and extended without being a 1-in-a-million
programmer/mathematician/genius. Add new algebra, but document it.
Change old algebra, but document it first. Expose the thinking as
well as the code. Whatever gets written will IMMEDIATELY become a
maintenance task for the future. Communicate the ideas.

Tim