RE: [Axiom-developer] document, graphics

[Page, Bill]

On Tuesday, June 28, 2005 7:32 PM Bob McElrath wrote:

> I would like to separate this discussion for the purpose of
> clarity. First, I have been discussing mostly document processing,
> and interaction with mathematical output.  As I've suggested,
> I think this can be accomplished with an absolutely minimal API:

I think this is a good design. :)

>    AppendInput(input)

What is the size of the unit input transaction? Would one Axiom
command per transaction might be too limiting? Of course it
might consist of multiple lines even for a single command. But
what about multi-line function definitions? What about compiling
SPAD code, versus interactive use? Should we treat a SPAD
compile as a unit transaction?

What about Axiom commands that do not increment the line counter
such as

 )set output autoload off

Should we prevent the gui user from accessing controls that
might interfere with the communication and state?

The API should always accept input without blocking. It will
perform buffering and queuing when Axiom is busy.

It will be important to design the API code carefully so that
it each transaction uniquely identified.

>    ModifyInput(line, newinput)

I assume you mean replace input line? (no need for API to computer
deltas) Ok if transaction size is small (one command only). At
this initial stage I think we should assume that editing will
remain basically a text editing problem, editing Axiom commands
for output and editing native HTML with jsMath extensions for
static text.

Maybe a little later we can play with HTML WYSIWYG gui editing
interfaces such as provided by Mozilla and/or javascript like
Kupu. I tried Kupu on the MathAction wiki but the embedded
LaTeX really makes this impossible. With a lot of work in
might be possible to use Kupu and jsMath together but for now
basic text-based editing seems like the best place to start.

>    GetOutput(line)

What happens if the output is not ready? Again non-blocking, gui
gets a "not ready" reply?

>    ComputationStatus()

Perhaps a timing and synchronization function in the API could
make this more efficient.

     WaitForOutput(line)

Some protocols might make use of "call backs" from the API back
to the browser. I think we should avoid this if possible.

I suppose the computation must be interruptible by the user.
Also in some cases (used to happen in Maple a lot :) the
following might be necessary

    KillAxiomAndRestart()

And maybe two more editing functions?

    DeleteInput(line)
    InsertInput(line, input)

Of course these could be implemented by ModifyInput but separating
them makes sense for efficiency.

Tracking all these things while performing some variation of the
LRE algorithm, sounds like a substantial coding effort. But I really
like this separation of function and isolating the browser from the
million little details. The idea of keeping the gui side as simple
as possible is the right approach. This also maximizes the benefits
since this API might very well be used by more than one interface -
for example TeXmacs would also greatly benefit.

The new TeXmacs/Axiom interface program on Windows is written in
C and makes extensive use of threading. Besides resolving timing
issues, it also does some significant recoding of the LaTeX
output (line folding) and accepts a limited set of commands to
control the state of the interface. I think LaTeX line folding
will remain an issue even with jsMath and MathML. Both MathAction
and the TexMacs interface make use of some fairly old and intricate
C code to do this manipulation. It works well but it is a bit
over due for some maintenance and code upgrading. I assume that
the new API will continue to do this LaTeX line folding?

> where the GetOutput call returns an sexp or xml document
> containing no code, but hints such as "sumterm1", "paragraph",
> "type".

The more I think about it, the less I think modifying Axiom to
produce a new output format is not a good idea for this project.
I think it is necessary to start setting some reasonable attainable
goals for the work over the next 3 - 4 months. I strongly suggest
that we retain the LaTeX output generated by Axiom for this stage
of the work.

I also think that attempting to use a lot of new XML to generalize
things is not a good idea at this time.

I recommend staying as close as possible to XHTML. Embedding
LaTeX as per the jsMath <span class=math> and <div class=math>
approach (in order to conform with as much as possible. LaTeX
embedded in XHTML still presents some coding challenges. To
minimize unexpected parsing failures CDATA bracketing might be
a good idea.

> Dependency tracking/re-evaluation can be handled inside Axiom.

I would say that Axiom should be used where possible with a
minimum of new coding. I expect a little archeology might dig
up some very useful pieces of interface code. I recall some
specific conversations with Mike Dewar regarding the extensions
that NAG coded in Axiom to support the Texplorer browser interface.
I am sure that most of that is still there but poorly documented.

I think however that a substantial part of the re-evaluation logic
will have to reside in the API. From my point of view, because
Axiom is so large, complex and largely undocumented, and the learning
curve is very high, I think making major modifications to Axiom
is just beyond the scope of this work.

> The last call ComputationStatus should return a list of lines
> for which computation is underway or need re-computation.

These could be used for worksheet status markup on the gui
display right?

> The second part of the discussion is non-mathematical input/output.
> e.g. graphs, diagrams.  Can we do the same thing as for mathematical
> output?  (e.g. the output contains only data structures and hints, and
> no code?)  This requires a graph renderer/manipulator which understands
> the underlying data structure.  (e.g. 1-dimensional point set,
> 2-dimensional graph, 3d point set with mesh, etc) "understanding the
> data structure" is a separate problem from "communicating the data
> structure" since the latter can be simply marshaled into an sexp
> or XML data structure.
>
> Furthermore "draw" should really be an alias for "give me a point-set
> approximation".  Axiom generates a point-set.  The *user-interface*
> must decide what to do with it.

I am not convinced that a "point-set" language would be adequate
given problems of resolution, scaling, rotation etc.

> Axiom shouldn't care whether it ends up in a pie chart or bar chart.
> These are user-interface concepts, not mathematical concepts.

This sounds very hard to me. I know that both Maple and Mathematica
have spent a lot of effort doing this sort of work over many years.

But I agree in principle if you are implying that it might be
a good idea to consider a modular approach, i.e. make it possible
to use a variety of external tools from gnuplot to fancy VRML
and large-scale emersion environments. If we can spend some time
to implement some of the new web standards here for graphics,
I think the pay-off would be great in the long run.

> So, I seem to be convincing myself that I want a back-end which
> knows absolutely nothing about user interfaces, but can serialize
> its output in either sexp or xml.

It makes me nervous the more generalized you make it sound.
I think it is important not to be ambitious. Instead we should
be able to make the most ground by sticking close to ground :)
We should do thinks that we now work now. Get a complete working
system for initial alpha release available as soon as possible.
And then work incrementally at replacing the kludge parts with
more well thought out code. That's the way it seems to go in
open source most of the time anyway, right?

I would even go so far as to suggest jumping immediately on to
your Tiddlywiki prototype as a good place for a running start.
(Great work by the way, Bob!) I think getting started fast with
our vision set at this high level is much better than continuing
to worry about low level details and other best practice issues.

Kai Kaminski address@hidden wrote:
> >So you're proposing *three* applications?
> >1) axiom
> >2) sexp -> xml converter
> >3) web browser/javascript
> >
[...]
> Even in the beginning part 2) will be more than an sexp-to-xml 
> converter. Take plots as an example. Axiom supplies a list of points 
> that have to be connected with straight lines/splines/whatever. Then 
> part 2) generates SVG output, which isn't too hard but not trivial 
> either.

I think doing this much with SVG is a good idea.

> Suppose the plot contains some text, like a title or a description.
> We can't just stuff the text into the SVG file, because in XML
> certain characters have to be escaped. I don't want to implement
> all this in Javascript. But I don't want to have an SVG generator
> in Axiom either.

Your approach makes sense to me. In fact that is more or less the
way Axiom does it now. There is an Axiom-side component of the
graphics and then there is a separate graphics process communicating
via sockets that renders and manipulates the graph. Ideally
the rendering and manipulation part can be absorbed into the
new gui API layer. And then you need to learn in great detail
how the existing Axiom-side code used to talk to that external
process. Most of this is just a matter of patient software
archeology. But since all of this is working on the Linux platform
at least the code is not cold!

> If there is not an svg generator in axiom, what format should
> axiom present?

We need to check out exactly what it does now. I think the best
approach would be to change the Axiom side as little as possible.
Then the problem becomes emulating the graphics processor part
in a portable platform independent way. From my point of view
if this is one of the only major accomplishments of the summer of
code project, then I would still be very very happy.


> Suppose we want to support LaTeX as an input syntax for HyperDoc
> pages or something similar (Martin Rubey wants this, as I 
> understand). Do we want to put a LaTeX parser into Axiom? Do
> we want to implement one in Javascript?

No! I think that if general LaTeX to new HyperDoc is really
desirable, it should be done by stand alone tools, as much off
the shelf as possible. Ultimately we should depend as much a
possible on common HTML and CSS for the document formatting
and layout.

> This is fundamentally impossible because LaTeX is mathematically
> ambiguous.  e.g.
>    \int_a^b x d x
>    \int_a^b d x d x
> 'd' is a constant.  For that matter presentation MathML is also
> mathematically ambiguous.  (hence the creation of content mathml
> and OpenMath)

This is a different issue.

> ...
> We should restrict javascript to solely interface elements, and
> not allow it to do any mathematical processing.

With the exception of jsMath prior to MathML, I agree completely.

> ...
> One wouldn't need AJAX for help pages...or even file access as
> I have used in my example, but rather just <a href="file:///...">.

Remember that some browser pages allow for live tutorial kind of
interaction with Axiom. Static web pages are not sufficient for
this purpose.

> Re: "new hyperdoc page".  For the time being I think we should
> concentrate on all new hyperdoc being in the current pamphlet
> format. For this interface we can convert to HTML.  In the future
> we can contemplate converting pamphlet -> html en masse.  But
> for now let's keep an HTML UI document separate from a pamphlet.
> We should engineer things so that we can do a trivial HTML UI
> document -> pamphlet conversion.  Javascript is in the document
> for UI elements, but would be stripped for pamphlet conversion.
> A proper pamphlet representation should not depend on javascript,
> or running any code other than what can be delivered to the axiom
> engine.

Don't confuse the hyperdoc format with the literate programming
pamphlet files. Right now these are two rather different variations
on LaTeX for quite different purposes. It is true though what you
imply using pamphlet format for maintaining the hyperdoc files
might not be such a bad idea *if* we can reliably and automatically
convert then to the actually presentation format used on the
new AxiomUI. This sound like a fairly big job to add on to what
is already a big job.

> ...
> What I am saying is equivalent.  But you're forgetting that
> the hyperdoc pages are built as part of the build process.
> So the "code in the middle" is essentially the build code.

Could you describe this process in more detail? I don't
know much about it. But I had assumed that pamphlet files,
if they are used at all for the hyperdoc files, would just
be passive containers.

Of course Tim is the guy to talk to about all this!

> ... let's keep "pamphlet" as our primary format for the
> time being, and convert it.

Convert it during the build and install process? Continue
to maintain it in the format used now. Right?

[re: documentation]
> >The only thing missing from this idea is "search".
>
> Good example. Search is complex. Do we implement that in Javascript
> or will the search code be part of Axiom? If we have a process
> between Axiom an browser we can just put it in there.
>
> Neither.  It is a user interface issue, not a mathematical issue.
> It should be in the user interface, somewhere.  (though god no...
> not in javascript)  An XPCOM component could do it.

This is a web browser/desktop environment so why not use one of
the free desktop search tools?

Some quick references:

http://nutch.sourceforge.net/blog/2005/02/open-source-desktop-search.html
http://wiki.apache.org/jakarta-lucene/PoweredBy
http://www.activemath.org/
http://labs.commerce.net/wiki/images/0/06/CN-TR-04-04.pdf

I really like the idea of these sort of tools being integrated
with the rest of the web but able to be customized for individual
researchers and Axiom users. The web is the only way to fly,
even when it's just that 10 gigabytes of stuff sitting on your
desktop ...

> Also note that one could generate an index file with words and
> URI's. This could be parsed by javascript to make an efficient
> search engine. (But, all the logic has been moved to the build
> system which generates the index file)

No, no. I would stay far away from this stuff. I think that
just as with everything else, we should "collaborate" with
other open source developers and integrate major components
like this where ever possible.

[Thanks for the great references to work on SVG. :]

> Keep in mind that SVG+DOM has all the elements for a user interface
> to graphs.

Yes, I think that this is probably the right approach for at
least the 2-d graphics. 3-d interactive graphics might be better
handled as a later enhancement. There is still a possibility in
this case to make use of the prior work by NAG on the OpenInventor
VRML interface. This was the graphics interface that was used
with the Techexplorer browser on windows. So I think a lot of
the code must still be in there somewhere.

Regards,
Bill Page.