Re: [Axiom-developer] Lazy re-evaluation (was: More AxiomUI)

[William Sit]

"Page, Bill" wrote:
> 
> On Tuesday, June 21, 2005 5:05 PM Bob McElrath
> 
> > Kai Oliver Kaminski address@hidden wrote:
> >> What kind of dependency tracking would you need? Just within a
> >> single page? For multiple pages? Send me use cases, tell me what
> >> you need.
> >
> >I've been thinking about this a lot lately.
> 
> Although I have not played with this much but both Tim Daly and
> William Sit assure me that the old Axiom Hypertex browser
> already implements this kind of dependency tracking. That is
> apparently the purpose of the \free{} and \bound{} attributes
> embedded the Axiom commands contained in the hypertex pages.
> 
> I would prefer to call this something like "lazy re-evaluation".
> 
> I think it is notable that neither Maple nor Mathematica have
> lazy re-evaluation in their standard worksheet interfaces.

I am not sure what you meant by "lazy re-evaluation" here (or it could be my
ignorance about the concept). Mathematica has two modes of assignments:
SetDelayed (:=) for functions and Set (=) for immediate evaluation and so does
Axiom: macro (==) and assign (:=). Both SetDelayed and macro can be used to
define *any* identifier, not necessarily a function (just think of it as
function without arguments although there may be a minor difference: "x == ..."
instead of "x() == ..."). Each time an identifier defined using SetDelayed or
macro is referenced, it is recomputed in Set or assignment (and in SetDelayed or
macro when these are computed), and this occurs recursively until all
identifiers at the "leaves" are resolved with Set or assigned values. The
SetDelayed or macro defined identifiers have no "values" associated and the
values are computed on demand. Do you consider this delayed computation lazy
re-evaluation? [I put "values" in quotation because the word value can have
different meanings at different levels; at a functional level, the SetDelayed or
macro defined identifiers of course have associated "values".]

The above delayed evaluation (which is automatically evaluated when needed) only
occurs *after* the SetDelayed or macro definitions have been executed earlier
(known the Axiom's run time system). This is different from the hyperdoc
browser, which may have only *read in* the input file, but *not yet* run any
assignment (delayed or otherwise). This is like in Mathematica when the you read
in a Notebook, commands are not executed (unless marked as initialisation cells
and the system set to automatically run these on loading). Now if a user
*skipped* some assignments (delayed or otherwise) that, say, defines x, y, and
*jumped* to an assignment for z that involves x, y, normally in the Axiom
interpreter, the x, y would be treated as undefined symbols
and perhaps cause either a syntax error or a wrong evaluation. In hyperdoc, the
system will know whether x and y are free or bound (because these are declared
in the input file when it is built for hyperdoc from regular input file), it
knows to search for their definitions and execute those assignments. 

So the hyperdoc action, as I understand it, is unrelated to "lazy re-evaluation"
of the first paragraph.

>  [spreadsheet interface discussion snipped]

> Unfortunately the authoring of hypertex pages in Axiom is not
> an easily accessible user feature, so in spite of having the
> possibiliy of lazy re-evaluation, right now it only applies to
> hypertex browser pages. It does not really provide this as a
> "worksheet" option as such.
> 
> > Mathematica seems to keep an array of input and output.
> > (In[] and Out[]).  If we treat these as simple variables,
> > dependency tracking would be as simple as keeping a linked
> > list among variables.  (e.g. a "has" or "indeterminants"
> > function)  In addition to that each variable would need a
> > "modified" flag.  Any time a "modified" flag is toggled,
> > axiom could traverse the dependency tree and mark outputs
> > with a "recompute" flag.
> 
> Of course Axiom also keeps a list of inputs and has a fairly
> sophisticated )history command.
> 
> Can \free and \bound names really be determined automatically?
> I think it is awkward to require the use to declare them
> separately the way it is done now in the hypertex browser.
> Perhaps this is possible with the help of some new code in
> Axiom itself. 

As you know, nothing is *automatic* unless the automation is designed into the
system using some convention or protocol and people follow them. In Axiom, there
is the compiler and there is the interpreter. The hyperdoc dependency resolver
enabled files (if I may call it that, an example would be odpol.htx) is
generated (automatically as far as authors of constructors are concerned) from a
source file (say dpol.xmp) created by the authors, but in principle, this source
file could have been extractable from the spad file or pamphlet file. The
convention is simply to have the documentation as text (with some embedded
hyperlinks), the input to the interpreter as -- comments after a key line "--
Examples" placed after the ++ comments for a function. In particular, no
declaration of \free or \bound is needed. During 1990-2, Bob Sutor was working
on the Axiom book and the .tex files were used for the book as well. It was, if
memory serves, experimental at the time and the .xmp files were basically
extracted from the spad file by deleting the code except for signatures, ++
comments, and examples. I remember I thought this was so cool I actually
included "-- Examples" in several spad files I wrote. However, these were not
incorporated into the hyperdoc browser database or the book (perhaps due to
space and time limitation of the book).  From the .xmp files, I think the
regression testing .input files are generated as well (correct me, Tim, if this
is not right; there is a odpoly.input in the distribution), and these .input
files probably also support the hyperdoc examples.

You can test the odpol examples in the hyperdoc browser by going to
Examples->Domains->OrderlyDifferentialPolynomial.

The file odpol.htx is now generated by ht.awk (I don't know where this is, may
be the file is $AXIOM/src/hyper/ex2ht.pamphlet) and renamed
$AXIOM/mnt/linux/doc/hypertex/pages/ODPOL.ht (see also ODPOL.pht, a pamphlet
version with output included). Tim added internal links. The file odpol.input is
in the $AXIOM/mnt/linux/input directory.

Unfortunately, I was not able to locate any dpol.xmp file from the current Axiom
distribution (Feb, 2005). I was able to find in my own archive an older
(partially damaged?) version of dpol.xmp (dated 9/13/1991) which contains
examples for other related packages as well, and what seems to be the current
version of odpol.htx (dated 2/7/1992), and also a tex file odpol.tex (dated
2/18/1992) which is a tex segment for the hyperdoc with Axiom outputs. These
two, and others were generated automatically (Bob Sutor, I believe, was the one
who implemented this, but I do not know the command/program that did it). The
two files do not match, but do give the flavor. I believe Tim must have the
"current" version. There should be a directory containing .xmp files.

So any package can have such a .xmp file extracted and converted into the
hyperdoc dependency resolver enabled .ht file.


> Should this be a front-end browser function or
> a back-end Axiom function. For example a function in Axiom
> might be able to return the list of input history numbers of
> those commands that need to be re-executed. 

This is unreasonable. The purpose to edit earlier input lines may also rearrange
the order of execution of the lines. Only the user knows what order of execution
is needed for his/her purpose. However, as you probably know, you can use:

)history )write filename.input

to capture the input lines (if you have ")history on") and edit it.
However, the resulting .input file captures all input lines, including all the
ones that did not work due to typos or syntax, etc, and so needs to be cleaned.

What is desirable in the interpreter interface, is the ability to edit previous
input lines in a 2D way. Mathematica allows it and makes it very easy to do
quick exploratory computations during the development phrase. The final order of
execution can be done with cut and paste right there on screen in the FrontEnd
and there is no need to save the history into a file first (because the
interface *is* the editor as well). Maybe an emacs or TeXmacs interface can do
that?

> These could be
> mapped to corresponding output areas in the browser.

A browser (web browser) is not a text editor (yet). In fact, it is probably
undesirable to make a browser into an editor. As nice as Wiki is, one still have
to edit the source text, rather than the rendered text. In Mathematica, this
would be like requiring editing the Notebook file in text mode (which one *can*
do, even on a cell-by-cell basis). I can't imagine editing XML in text mode (I
have tolerated editing html, including those generated by MS Word or including
Chinese, but I think direct editing XML is not for me).

Speaking of XML, Microsoft "opens" up its XML technology, drawing a lot of fire.
Do a google on "eweek XML Microsoft open source" for more.

> > [clip nice example]
> 
> > A user interface could choose to highlight out[1] as needs-
> > recomputing, or just go ahead and do it. (spreadsheet-style)
> 
> I think it would be pretty neat if the browser could do such
> highlighting and very useful if with one click you could ask
> Axiom to minimally recompute just those comands that need
> re-computing. This should be quite easy in principle using the
> kind of AJAX techniques that Kai mentioned elsewhere. The hardest
> part would be writing the Axiom code that dynamically computes
> the dependency tree.

Am I missing something here? Can't one simply use == and := to achieve the same
thing already? As I commented above, it is *not* Axiom's job to decide for me
what needs recomputing and what does not. (Think about all the "automated"
choices MS Word provides -- I spend more time undoing them than actual writing).
I decide by correctly using == and := and recalling inputs.  What may be needed
and would be extremely helpful in terms of user experience, is again what
Mathematica allows: 2D editing, with multiple selection of cells, not
necessarily adjacent, and then one shift-enter does it. I would love to have
Mathematica FrontEnd working for Axiom (the best of both worlds). There are many
many features of the FrontEnd that are great and unparallelled (to name just a
few besides those already mentioned: all Notebooks are ASCII, include editable
input and output, graphics, incorporated style sheets,
independent of platform or operating systems, can be converted into html, tex,
etc; the entire Mathematica documentation available, with live editable
examples, and programmable using FrontEnd commands).

William