[Axiom-developer] Re: Axiom tutorial by Bill

[root]

Good morning, Joris.

You're ahead of the curve on this one. I haven't had a chance to view
the tutorial under TeXmacs yet. I'll download it today and review it.

re: tex macros

One of the things to think about is that the tutorial was written to
run in Axiom's help system called hyperdoc. Hyperdoc has several 
features that are enabled by embedded tex macros. All of these are
in the axiom style file. Using tmdoc would break things in hyperdoc.
We'll have to work out some compromise.

It will allow examples to be clicked on and executed. 

It allows "constraints" between lines. That is, if the example that is 
chosen requires previous examples then the previous examples will be
automatically run first. If you try to execute, say, a draw command
and the function defined in the draw command is defined elsewhere
you can specify in the tex macros that the definition must be run first.

It allows navigation (thus the random-looking "help", etc).

It allows embedding the live graphics image directly into the page.
Thus if you execute a draw command the image will show up in the page.
But if you click on the image it becomes a separate window on the desktop
and can execute live (e.g. rotate, scale, print, etc).

It allows "tear-off" pages. That is, you can click on a page and move
it to the desktop as a freestanding page. It would be like clicking on
a mozilla tab and opening it a new desktop window.

Almost all of this is supported by inline TeX macros.

Several new facilities are planned such as the ability to follow
citations in the text, dynamically extract, compile, load, and execute
the code in a pamphlet, dynamic "booklet" construction by adjoining
sections from pamphlets, semantic search (so you can find Abelian as
a concept, as a math definition, as code implementation in a particular
domain, or as standard text in pamphlets and booklets.

re: buggy tex/latex

If you find buggy examples of tex/latex please send them to the list.
I know of one bug due to escapes that I'm chasing on another queue.
I'm documenting how I debug it in the DevelopersNotes file so developers
can see some of the debugging tools. It is listed as an open bug.

I never use tex/latex output so I'm unlikely to see the bugs.

re: trailing ->

We could probably implement a "set prompt" command that would allow
you to set the prompt to anything (including nothing). Would that help?

re: the more general interaction of TeXmacs and Axiom

We've had many discussions about this subject as I have a near religious
belief that documentation is vital. We're not using the computer to do
anything more than a fancy typewriter at the moment. That's fine for now
but it will be positively primitive in 30 years. I've attached parts of
other discussions here just to recover them from history.

I'm not sure how far TeXmacs is willing to change to support Axiom.
Clearly most of the technology we develop over time will not be specific
to Axiom (e.g. running embedded code in a pamphlet could be Maple code).
If the machinery is done correctly then the current Axiom would just be
a plugable "engine". A lot of the ideas would be useful in TeXmacs.

Sorry, I'm off building castles in the air again. I'll stop.

Tim

> I am going through the Axiom tutorial by Bill, which looks really nice,
> and have a few comments:
> 
>   1) I very strongly suggest to organize things in the same way
>      as we do for the TeXmacs documentation (i.e. use the tmdoc style).
>      This should still not be hard to change at this moment,
>      because you also nicely cut the documentation into small pieces.
> 
>      The advantage of tmdoc is that it separates content from linking
>      information: the main documentation should contain few hyperlinks
>      (no explicit top, previous, next links) and meta-documentation files
>      contain information about the logical arrangement of the individual
>      files (this also allows you to share content between several books
>      with potentially different orderings of sections).
> 
>      In the future, the tools around tmdoc should be further develop
>      (you may participate by the way: this is not very difficult stuff).
>      For instance, we already support the automatic generation of
>      one large document with all files and the creation of web-sites.
>      We plan to add top/previous/next links on request, and wiki-like
>      back-links. So: by trying to conform to the tmdoc drd, you will
>      benefit from all these possibilities...
> 
>   2) I need to implement something which allows you to remove
>      all trailing ->'s in embedded axiom sessions.
> 
>   3) I notice some "strange styles" on the bottom line from
>      time to time, like "help axoim-tutorial axiom-tutorial axiom"
>      (see oneStep06.tm). You should make one style "axiom-tutorial",
>      based on "tmdoc" and using the package "axiom".
> 
>   4) Functions with arbitrary numbers of arguments will be obsolete
>      in the next version of TeXmacs, so you will need to rewrite
>      your customized menu macro.
> 
>   5) The TeX/LaTeX output of Axiom is quite buggy from time to time.
>      Some effort really has to be spent on this issue.
> 
>   6) It would be nice to integrate the Tutorial into the TeXmacs
>      interface for Axiom. Do you already have some ideas on
>      how you want to do that.
> 
> More later, but let us first discuss the above points.
> 
> Best wishes, Joris

======================================================================

I like your idea of "synopsys" and "algorithm" tags.  Clearly the
pamphlet files need structure.  However I'm not sure that the chunk
tags are the right place for the structure. The chunk tags are 
intended to quote and embed code in a tex document. The booklet
extension allows the code to be in files which are "included" but
still has the idea that they embed code. We're beginning to evolve
the noweb idea. David Mentre took the first step with the booklet
code.

> Bertfried Fauser wrote:
> Regarding my current view on mathematics, it is far from beeing a static
> sort of things. Many people do believe that mathamatics is given as-is and
> has only to be 'described' or 'explained'. However, I personally expect a
> great reworking of many areas of mathematics in a more 'categorial' way.
> Any documentation of a project like AXIOM should be capable of such
> reorganization, and you do indeed think about this.

Actually, Axiom's math evolves. Bill Sit has given me a newer version
of the PLEQN domain that includes later work. Several other people
have indicated that they also have later, improved work. I'm
struggling with the idea that Axiom should have an embedded CVS so you
can add new work while still referencing the old. And CVS solves the
problem of authors taking different directions with the same
mathematics. But I'd have to change the build structure to take this
into account so this idea is a long way off yet.

What you appear to suggest is that portions of the document might contain
tags, lists, or markers (TLM) that indicate the intended use of the
material.  The machinery for this would be easy to add. We simply need
to have a parameter on various tex tags (e.g. \section[synopsys]{The
Section}) What isn't clear, as you point out, is the meaning
associated with the TLMs. Perhaps a standard portion of the pamphlet
could exist (similar to an index) to carry this information. I have
a "concept index" for commutative algebra which I plan to use to
structure the booklet files once they come into existence.

My thoughts on the subject, clearly open to debate, are that the 
pamphlet files contain information that the author can structure
to explain a particular point. Think of these files as "technical
papers" which potentially contain code. A technical paper has a
certain assumed, but not required, structure by convention. It has
an abstract, background, new results, summary, and bibliography.
I think the pamphlet files will evolve structure by convention also.

This raw information can be used and viewed in many different ways.
My thought was that "booklet" files would be the organizers of information.
So to address your example of sophistication level I'd expect an author
to write a booklet file that includes portions of pamphlet files. Simple
booklets such as an intro to linear algebra would only include text from
the examples portions of pamphlets. Complex booklets about lie algebras
might include portions of the theory sections containing theorems. So
booklets are intended to organize the pamphlet information for a particular
purpose. Pamphlet files are for organizing raw information about the code
and its supporting mathematics. I'm not sure if this is (a) a clear division
of responsibility or (b) a workable solution to your issue. It needs more
discussion.

> Bertfried Fauser wrote:
> My own experience shows, that documentation of code takes away 2/3 of the
> total amount of work needed and only 1/3 goes into coding and testing.
> Waht is clear to oneself has to be described very precicely in a
> documentation and there has to be a mechanism guaranteeing that code and
> documentation are recent and mutually related. Perhaps this is already
> guaranteed by using literate programming. However, there is a notorious
> tendency to make small hacks and 'forget' about documenting them, since
> one thinks about doing this in the next major realeas.... which never
> happens.

Knuth and Dijkstra both made this observation and both decided that
literate programming was the technique most likely to attack the issue
of documenting the code. It is ultimately up to the programmer to document.
I'm hoping, by force of convention, to convince Axiom programmers that it
is worthwhile spending 2/3 of their time documenting. The saving grace
is that, at least in mathematics, you get professional recognition for
writing up your results so you can, at minimum, attach the technical
paper to the code and call it a "pamphlet". That allows later authors to
better structure the mixture. If we succeed in keeping the theory with
the code it will be a major improvement over the current situation. Of
course, I'm not satisfied with just mixing the two. I want to exploit
the collection and make it useful for broader purposes.

When the alpha version gets uploaded look at
src/algebra/dhmatrix.spad.pamphlet. I've combined the theory with the
code. Denavit-Hartenburg Matricies (DH Matricies) are 4x4 matricies
with special structure that enable them to describe rotations,
translations, scale, perspective, and skew operations. This domain was
used to create the graphics in the glossy pages of the Axiom book.  I
wrote the original spad code essentially without comments. The theory
comes from Richard Paul's Ph.D.  thesis (which he gave me permission
to use). I'm now trying to atone for my sins and document the domain
properly. Since this code will support graphics I'm hoping to use it
as an example domain to exploit ADVI's ability to embed active graphics.

ADVI seems like worthwhile technology as they have found a mechanism
that allows authors of tex documents to embed information into the
dvi file that ADVI can use. This allows Axiom to perform all kinds of
interesting magic which opens up new ways to construct "books", booklets,
pamphlets, slides, technical papers, etc. And it uses "standard technology"
(since Knuth anticipated this in the dvi file format).

The alpha release of Axiom will use an axiom.sty file that will eventually
allow us to embed ADVI information. ADVI has shown what's possible but we
need to do much more.

=======================================================================

It is important to get the documentation available quickly but it is
more important to structure the documentation to support the system
as a whole. I'd like to see Texmacs and ADVI support pamphlet files
in their native format. So one of the delays has been that I'm trying
to understand the details of ADVI and what changes might be required
to support the literate programming style.

Please try to look "into the distance" with the documentation and
Texmacs. Once the documents get released "into the wild" there 
will be great resistance to changing it because people will build
tools (like the axiom-texmacs interface) that they will refuse to
change for "legacy" reasons. We've already seen this with the 
axiom-texmacs "prompt" issue. NAG's axiom hasn't been available for
2 years and still people won't change. 

The documentation structure is critical to the long term success of Axiom,
a point I've beaten beyond the grave. But the documentation structure has
to support things like Axiom Journal articles, "drag and drop" literate
programs (where you drag a pamphlet file onto a running Axiom and it gets
compiled, the docs get put in the right places, the required references
get fetched and loaded, etc), pamphlets can be combined into booklets, etc.

Fifteen years ago we did hyperdoc which was a browser-like technology
with hyperlinking. This was years before Mosaic was conceived and it
was very creative then but it's dead technology now. What new features
will you create when you combine Axiom and Texmacs that will look
positively insightful fifteen years from now? What features are
lacking in all of the online documentation you've already seen? How
does having the embedded code and/or the ability to have "live"
documents change the game? I'm interested in ADVI because I can have a
document page that includes moving graphics generated in real time.
When we did the hyperdoc we set it up so that you could get a page
with a draw command which, when executed, launched an embedded
graphics page. That embedded graphics was "live" and you could rotate,
scale, shade, and print it. How can we do this in Texmacs?  

What will mathematicians need from the documentation 30 years from now
and what do we need to do now to support that? Can texmacs import the
axiom type lattice and "know" about the domains? Can it dynamically
build booklet tables of contents as you visit files (so I can create a
booklet by visiting a series of pamphlets (this would allow me to keep
a structured booklet of the math I've learned from the pamhlets.
Essentially it is a personalized book)). Can it "fold" and "unfold"
code and/or documentation? Can it follow references and bibliographic
links to load code and docs? Can I construct a course outline and let
texmacs find related material in the docs (e.g. a linear algebra
course)? If you look at the linear algebra course at MIT's OCW website
ask yourself how the documentation (tests, projects, video, course
outline, textbooks, etc) can all be combined in some form of pamphlet.
Can I construct a CAD-like graphic image of a bridge and attach both
the matrix mathematics and its associated documentation to points in
the bridge image? Can I structure the documentation so that the
documentation presented to the user is adjusted to his level of
expertise (e.g. a student discussion of matricies vs a research math
discussion of matricies)? What if the notion of files goes away and
all of Axiom is just one big, coordinated document (that happens to
have execution sematics for portions of the graphics or the code).
How can I think about, navigate, modify, and maintain a terabyte sized
"document" that is Axiom? How do we integrate a semantic web into the
documentation so that the concepts are automatically classified into a
proper hierarchy (e.g. KREP, a knowledge representation system). A
KREP-style semantic web would allow a user to navigate by concepts
rather than text. KREP is 20 year old technology and we should be
using it already. The "next internet" project is looking at building a
global semantic web. What will it mean to Axiom if work you do today
is instantly classified and available worldwide tonight?

So I guess I'm asking you to be innovative with the documentation as 
it is critical to the long term success of Axiom.

=======================================================================

Axiom needs the documentation mechanism to support research in a deeper way.

When researching a concept I generally have a pile of research papers
as well as my own notes and experimental code. There are also numerous
versions of these things including versions of test cases and results.

I've looked at some "outlining tools" that could have some helpful ideas.
In particular the ability to "fold" text so you can temporarily blank
out intervening sections is quite useful. I tend to do this a lot in
emacs by splitting the buffer and having two views of the same buffer.

But that's just a simple tool. We need to think deeply about the issue
of organizing (a really bad choice of word as is "capturing", "structuring")
the work done during research.

For instance, I'm working on the Hanna-Neumann conjecture. This involves
writing and running code to check for possible counter-examples as well
as exploring the boundaries of the problem (e.g. if a counter-example
exists it must have a certain form). To do this I generally write code,
try it and examine the results (possibly making a note to myself about 
some vital insight in some ephemeral form like writing it on my hand).
Next I'll try another slight modification to the code, etc. In a better
world the file system would keep all of my versions (the Symbolics machine
filesystem used to do this) as well as the attached notes. It also involves
none-machine interaction like conversations, blackboard work, and email
discussions. My research rarely ever occurs in the office because I can't
get contiguous blocks of quiet there.

I've been muttering about the "30 year" design goal and what we how we can
innovate now to support researchers in the future. 

There is a book called "The Mind Map Book" by Tony and Barry Buzan.
They seem to have thought about this also and I need to read the book
with an eye to understanding their concept.