Re: [Texmacs-dev] Table converision library

[Joris van der Hoeven]

> Patches 1048 and 1071 are just regression tests for tmtex. So they are
> independant of the table manipulation library.
> 
> However I see a use for a library of tools for regression testing. In
> that case, the functions used to produce tm-tables for testing the
> converter are usable by any converter. It is important that the
> regression testing tools do not overlap with tools which do actual
> work, because it reduce the likelihood of common failure modes.

That is OK, but I would like to concentrate on the precise
functionality first.

> I think it is a bit early to set a definitive design for the table
> conversion library but I will explain the general idea behind patch
> 1051.
> 
> The tm-table-parser (currently tmtex-table-parser) uses an
> object-oriented approach to provide modular, thread-safe and
> referentially transparent parsing of table formatting for use in
> converters. It creates a table-parser closure given a table-default
> list and a table-definition.

If I understand you well, the idea of the parser is mainly to
increase performance and simplicity of some of the algorithms.
>From the user point of view, we are of course mainly interested
by routines for extracting information from tables (or constructing
tables for * -> TeXmacs converters).

> A table-default list is a list of 'twith' and 'cwith' elements, it is
> meant to hold the default table settings which are set by the macro
> expansion enclosing the table definition. For example 'block*' sets a
> global border and centered horizontal alignement for all cells.

Notice that the default table list should be provided for free
in most cases by evaluating the to-convert expression with
the exception of all keywords which can be converted
(cf. our previous discussion).

> A table-definition is a Scheme tree whose root is a 'tformat' or a
> 'table'. It is meant to be the value of a table macro argument.
> 
> The table information is accessed by evaluating table-parser closure
> with different arguments. The first argument is a symbol and denote
> the scope of the property being accessed. Different scope require
> different additional arguments. Currently existing scopes are:
> 
> Iterator scopes:
> 
>   rows: name -> list of values
>     Values of the named property as it is defined for whole rows.
> 
>   cols: name -> list of values
>     Values of the named property as it is defined for whole columns.
> 
> Synthetic scope:
> 
>   global: name -> value
>     Synthetic properties, as a boolean 'border' value, the number of
>     rows and columns.
> 
> Currently envisioned future scopes...
> 
> Physical scopes:
> 
>   row: number name -> value
>     Value of the named property for a given numbered row.
> 
>   col: number name -> value
>     Value of the named property for a given numbered column.
> 
>   cell: rownum colnum name -> value
>     Value of the named property for a given cell.
> 
>   table: name -> value
>     Value of a table property.

Yes, this distinction is OK.

> It must be noted that smaller scopes may provide information which is
> not available to wider scopes. For example a column may define a
> column-wide bottom border of "1ln", and one cell of this column may
> define a bottom border of "0ln".

Absolutely. It is very important to be able to extract properties
from wider scopes. For instance, when converting to LaTeX, you should
first extract the properties for each column (the last cwith which
covers the whole column) and put this information in the argument
of \begin{array}{...}. Next, this information may be overidden locally.

> The 'rows' and 'cols' scopes are shorthand for mapping the access of
> the property on all 'row' and 'col' scopes.
> 
> Properties are named by symbols. They match table properties used by
> the typesetter. Rows, columns and cells also define the extra property
> 'content' which gives the table data. Rows content is the table data
> as a list of lists in natural order, columns content is the table data
> in transposed order, cell content is the actual content of the
> designated TeXmacs cell element

Yes. Notice that you might also want to extract content and properties
simultaneously for certain purposes. In fact, the table paradigm is
very rich and we will need to play with it...

> Handling of subtables and decorations is not yet defined, but I
> believe the current design to be generic enough.

Please don't bother about that yet. We first have to understand
the semantics of the tformat/cwith construct better.
This really is a new, and interesting, level of abstraction which
is not really present in XML.

> There could also be some accessors for grouping scopes, which might be
> useful to carry the meaning of cwith on ranges which are not simple
> rows, columns. But maybe that would only be really meaningful when
> TeXmacs has a real support for row groups and column groups.

Notice that we also have subtable-scopes.

I also agree that we need better support for row groups and column groups.
A good example of a situation where this is needed are numbered equation
arrays. One would like to be able to have properties or even macros
for saying that a row should be numbered or not.

> Maybe that is all crap. If you think so, please propose another design
> which provides the same level of encapsulation and extensibility.

No, I think that you are rather close to my views on the subject.
I would like you to regroup the code which is independent from
tmtex in a new file though, so that it will be easy to study it.
Also, it would be good to clearly separate the different parts of
the library (parsing, extraction, etc.).