[Texmacs-dev] DOM and TeXmacs roadmap (was: getting involved into to Scheme)

[David Allouche]

On Wed, Jul 09, 2003 at 11:55:39PM +0200, Immanuel Normann wrote:
> I'd like to get familiar with Scheme as TeXmacs extension language. I 
> already know Scheme but not how it is used in TeXmacs and there is not 
> to much documentation on it. Also I looked at the source 
> TeXmacs/progs/... But then i decided to ask you where the best starting 
> point is!

That is a good question...


> My interest is to figure out how much the already implemented features 
> of TeXmacs Scheme are suited to manipulate the document in general.

Yes and no.

Some people say "you can process the document as scheme expressions so
you can do anything", other consider that the lack of complete and
conventional foundation libraries, and the work-in-progress
not-quite-there-yet state of the structural processing libraries is
still a problem.

The bottom line is that you can do A LOT of things, but it may require
either a lot of coding or a extensive knowledge of texmacs abilities,
and maybe both.


> Is there something comparable (planned) like a "Document Object Model" 
> (as e.g. http://www.w3.org/DOM/ but adapted to Math content)?

Yes and no, again.

The following information was gathered from my understanding of what
texmacs does and where it is headed and a reading of the following
page:

    http://www.w3.org/TR/2000/WD-DOM-Requirements-20000412/#General

I have no practical or precise knowledge of what DOM really is, so I
may have made faulty assumptions.

DOM level 1:

  Structure navigation: can be implemented quite easily using a
    SXML-like approach to the STM (Scheme TeXmacs) document
    representation. All STM documents can trivially be considered SXML
    documents.

    Note: the STM representation of any selectable texmacs data can be
    obtained using Edit->Copy to->Scheme.

  Document manipulation/content manipulation: can be implemented in
    the current framework, the current work on converters and generic
    structural operators makes it a active field at the moment. One
    important (and currently unsolved) problem is the duality between
    the relatively static (but easy to use) STM document manipulation
    and the more complex (but more dynamic) manipulation of live data
    in editors. I will have a look at the DOM spec to help me in my
    quest for enlightenment.

DOM level 2:

  Event model: texmacs does not have a powerful enough event handling
    system to support this. Currently, events are separated in two
    classes:

    -- keyboard actions: can be redefined programatically under
       contextual constraints (validity of a scheme predicate).

    -- double click: can be handled by an ACTION primitive (requires a
       change in the document structure). The ACTION primitive
       executes a scheme expression when its typeset box is
       double-clicked.

    The event model is planned to be completely reorganized "sometimes
    soon", that is before one year. Eventually, the distinction
    between keyboard actions and mouse actions should go away. It is a
    good time to make well thought and pragmatic proposals. Saying
    "just do it as in the W3C spec." probably won't cut the mustard.

  Stylesheet object model: not in the scope of texmacs at all at the
    moment. We are currently discussing a way to extend texmacs macro
    expansions to support XML-like attribute nodes.

    In the long term, texmacs should also get a better separation
    between styling and typesetting which would allow for CSS-like
    (instead of latex-like) stylesheets. But do not expect this to
    come this year.

  Range model: TeXmacs has a "marker" facility which allows for
    position markers which are robust when the document tree is
    modified. So this part of DOM might be partly implemented. The
    "range of text rather than subtree of nodes" concept is not
    planned to be a core feature but it might be implemented as a high
    level library.

  Traversal model: hard to tell.

DOM level 3:

  It just had a very high-altitude fly over this one, but it really
  seems very alien to texmacs as it is today.


> A very simple case study that might illustrate the point:
> Suppose I have several occurences of the function symbol "f" spread in 
> the document. One of these occurences is may be within a definition. Now 
> I want to click on this "f" and that should have the effect that all 
> other occurences get red as backgroundcolor.
> 
> Is this doable with Scheme in TeXmacs?

It is doable, but it would require a number of constraints. I strongly
encourage all research in this domain because the structural operation
abilities of texmacs are really under-used. You might benefit from
collaboration with other users which also work in this field (Alain
Herreman?).

A way to do this would be:

  -- have the f in the definition be inside a special environment
     (macro) which expands (maybe among other things) to an ACTION
     box.

  -- the associated action would traverse the document and wrap all
     relevant occurences in a special macro (maybe called
     temp-hilight). Currently, the recommended way to do this is get
     the STM representation, do any operation you want on it using
     usual scheme programming techiques, and set the buffer to the new
     data (maybe saving and restoring the cursor point to preserve
     stability).

  -- provide a cleanup function to remove all occurences of
     temp-hilight, which could be invoked manually or when another
     hilight action is executed.

> How can I access nodes of the document tree?

Two ways:

  Using STM:

    (tree->object (the-buffer))
      gives you the STM representation of the current buffer text.

    (tm-assign '() (object->tree x))
      sets the current buffer from the STM data in x.

  Using the trees and editor commands:

     This is more complex: many primitives are involved (including
     tree access, seven editor primitives, and library editor
     commands) and many of them have very little sanity checking (it
     is very easy to segfault texmacs).

     Really not the recommended way to work unless you really have to.
     But it is sometimes required when the "stability" of the document
     is important. Here "stability" means reducing the re-typesetting
     time and preserving the position of the cursor and markers.

> How do I change their appearence?

By adding structure which affect the appearance. To be done properly
that often requires using a tailor-made "*.ts" file to define the
typesetter macros you need.

> How can I add or remove event listener to such a node?

As I have explained, there is currently no such thing as event
listeners in texmacs typesetters. You can get double click
functionality by using the ACTION primitive (see "changing the
appearance") and you can customize the keyboard mappings in such a way
that it only effects specific nodes.

  Note: in the next monthes, texmacs should be modified to support
  HTML form, this will require putting GUI items inside the document.
  Also, Joris plans to remove the distinction between document tree
  and GUI tree, so the EventListener feature might become a reality
  soon.

If you really have a need to extend the keyboard mappings on uniquely
identified nodes, you will need to add such unique ID information to
your document structure (the HIDE_EXPAND primitive would come handy
here).

-- 
                                                            -- ddaa