steps towards a Maxima interface in MathAction (was: [Axiom-developer] steps towards better TeXmacs interface)

[Page, Bill]

On Friday, August 26, 2005 12:33 PM Andrey G. Grozin wrote: 
> ... 
> Another free computer algebra system, maxima, has incorporated
> inserting of parametrized markers in the key positions of its
> output some time ago. Now it is easy to interface it to almost
> any external program: one just assigns appropriate values to the
> marker strings, and then maxima speaks the protocol required by
> this external program. Maxima developers called this "the transition
> from the stone age to the bronze age" (the idea is that using corba
> or some such thing would be the enlightment era).
>

For some time now I have been wanting to implement an interface
to Maxima in MathAction. I am not a Maxima user, although I have
installed it and played with it a little. I was hoping that someone
with more Maxima experience would come forward and offer to help
me to do this ... :)

Where would be the best place to look for a definition of these
parameterized markers in Maxima? What would you suggest as a
simple example of a program that parses Maxima output using these
markers?

The problem of interfacing with MathAction is not exactly the same
as in the case of TeXmacs. TeXmacs which starts a parallel external
session. It then sends commands, receives and formats output in a
synchronous manner while the user edits the document. In the case
of MathAction, the entire web page is edited in "text-mode" and is
only processed by an external program (such Axiom) once when the user
clicks 'Save' or 'Preview'. Since a web page consists of multiple
sections of alternating formatted text, LaTeX commands, and commands
to external programs, these sections must be collected in some
manner and then sent to the external program. The output of the
program must be parsed and separated into output sections that
correspond one-for-one with the specific input sections. Then during
the rendering of the page as HTML, MathAction replaces each of the
input sections with selected and possibly reformatted parts of the
output (e.g. LaTeX generated by Axiom).

In the case of both Axiom and Reduce I solved this problem by
constructing a .input file for each section and then running Axiom
(or Reduce) with a script that executes each .input file by a
series of )read commands. The output corresponding to each )read
command is automatically delimited by the Axiom (Reduce) prompt
sequence. So separating the output into sections is quite straight
forward. (But of course would be even easier if Axiom was specifically
designed to delimit the output in the way that Andrey suggests.)

The problem that I have with Maxima right now is that I do not
know how to run Maxima in this manner, i.e. in a "batch mode"
to process a series of input sections and then parse the output
into a series of corresponding output sections.
  
> Axiom is still in the stone age in this respect (a more structured
> and predictable input-output). I am now trying to move it to the
> bronze age. The first (and rather obvious) place is interp/i-
> util.boot.pamphlet I renamed the procedure MKPROMPT to MKPROMPT0,
> and inserted
> 
> $InterfaceStrings : Record(PromptPrefix:String,PromptSuffix:String)
>  := ['"[",'"]"]
> 
> MKPROMPT() ==
> 
> STRCONC($InterfaceStrings.PromptPrefix,MKPROMPT0(),
>  $InterfaceStrings.PromptSuffix)
> 
> This works, and my prompts are now in brackets (of course, this is
> just a test). The next step is to find all the other prompt-like
> things in axiom (i.e., output strings immediately preceeding user
> input). One of them is the exit confirmation, for example; I'm sure
> there are many others. Each prompt-like string should now be preceeded
> by $InterfaceStrings.PromptPrefix and followed by
> $InterfaceStrings.PromptSuffix . And this is only the first step,
> there will be, e.g., $InterfaceStrings.LatexPrefix and
> $InterfaceStrings.LatexSuffix and some other markers.
>

>From my experience with MathAction I know that it is important to be
able to efficiently and reliably parse the Axiom output to identify
and separate the LaTeX output $$ ... $$ from non-LaTeX output like
error messages and other informative messages, and also to identify
the Type: ... information. Axiom's current output format is just
barely able to support this. New $InterfaceStrings to support this
parsing would be very welcome!

> So, I have a few questions to the gurus:
> 
> 1. What is the correct file to put the definition and initialization
>  of $InterfaceStrings ?
> 
> 2. Is there some way in which this initialisation could be made
>  dependent on external conditions? For example, can I pass a
>  command-line argument to AXIOMsys, and check it in boot code? Or
>  check an environment variable in the boot code? At the moment,
>  I know only one way to pass some information to axiom:
> .axiom.input. But it would not be a good idea for TeXmacs, for
> example, to overwrite the user's .axiom.input (if the user has one).
>

My preference would be to pass InterfaceStrings via one or more
environment variables. Reading environment variables in boot code
amounts simply to calling the corresponding lisp system function.
 

> ...
> TeXmacs has some strange requirements of its own (e.g., using \* for
> multiplication).

This is not a "strange requirement" since one of the goals of TeXmacs
is to retain more of the mathematical semantics then what "normal"
LaTeX allows. This is the same goal that the developers of OpenMath
had in mind.

> Therefore, I want to leave the old TeX generation alone, and to
> introduce the new kind of output: texmacs. So that it will be
> possible to say 
> )set output texmacs on

I think that this is an excellent idea. A couple of years ago
When Tim Daly and I first discussed this, Tim provided some
notes on how the )set output commands were processed in Axiom.
If you are interested, I think I could probably find them in
the axiom-developer email archives.

> OK, I can easily duplicate tex.spad.pamphlet (should this be a
> new file, texmacs.spad.pamphlet, or I just double the contents of
> tex.spad.pamphlet ?)

I think a new file is better unless you find that there is a
large overlap in the source code, i.e. same set of helper
routines etc.

> I'll rename the domain TexFormat to TexmacsFormat, and make
> the improvements I want. But what to do next?

Since this is going to be specific to TeXmacs anyway, have
you considered using TeXmacs' scheme format instead of LaTeX?
This way you can avoid the complications of the TeXmacs LaTeX
conversion.

> My (very incomplete...) understanding is that I should insert the
> necessary pieces into setvars.boot.pamphlet and setvart.boot.pamphlet
> (based on the TeX pieces, with renamings and some small changes).

Tim described some of this in his emails of two years ago on
axiom-developer.

> But setvars.boot.pamphlet says that if I change the boot code (and
> I'm going to do this), I have to translate it to lisp and insert it
> into the same file. How do I do this, exactly?

Specifically in setvars.boot.pamphlet it says:

This file contains both the {\bf boot} code and the {\bf Lisp}
code that is the result of the {\bf boot to lisp} translation.
We need to keep the translated code around so we can bootstrap
the system. In other words, we need this boot code translated
so we can build the boot translator.

{\bf NOTE WELL: IF YOU CHANGE THIS BOOT CODE YOU MUST
TRANSLATE THIS CODE TO LISP AND STORE THE RESULTING LISP
CODE BACK INTO THIS FILE.}

-------

Boot is a compiler. I presume that that lisp code that you need
to insert into this file can be generated just by compiling the
new boot code (the way it is done now in the part of the Axiom
Makefile that builds INTERPSYS. You should be able to capturing
the lisp output in the NRLIB directory. Is that right Tim?

> ... 
> There is another possibility, which requires some thought. What if
> we send expression tree from axiom to TeXmacs as an S-expression,
> instead of serializing it via LaTeX? TeXmacs understands not only
> latex: fields, but also scheme: fields. Generating scheme syntax
> from an axiom expression may well be easier than generating LaTeX.
> Line breaking can be done on the TeXmacs side, where the necessary
> information is available (it is in principle impossible to tell,
> what is the width of a sub-expression after typesetting, without
> actually typesetting it). But this is, probably, a more long-term
> project. For now, I'd better keep as much as possible from
> the current interface, but make it more robust and less ugly.
> 
> What do you think?
> 

I like your last idea best. I think it would be a mistake to spend
too much effort simply tweaking the LaTeX output to make the current
interface more robust. The idea of having a )set output mode that
generates semantically rich S-expression output is very appealing
not only for use in TeXmacs but also for the work that Kai Kaminski
is doing right now with the new AxiomUI. I think that many of the
hooks that might be necessary to do this in Axiom must already be
present in Axiom since this goal is very similar to to goal that
NAG had towards the end of the commercial development of Axiom to
produce OpenMath output.

After the current (rush) phase of the Summer of Code AxiomUI project
is over at the beginning of September, I hope that Kai will have
a chance to discuss here more of what he had in mind of AxiomUI in
this regard.

Regards,
Bill Page.