Re: [Texmacs-dev] string encoding

[Joris van der Hoeven]

> > The < and > characters are used as a placeholder for extensions:
> > < followed by any sequence of non-<> characters, followed by >
> > is interpreted as one single character. This allows us to use
> > a potentially infinite alphabet. In particular, the TeXmacs encoding
> > will englobe unicode in the future, without being limited to unicode
> > (we do have some mathematical characters which are not in unicode).
> 
> In the light of adding support for XML: 
> 
> 1) How should a <char> sequence be mapped to XML (if the corresponding
> char are not in unicode)? &lt;char&gt; is not that readable. 

What we should start with is building dictionaries between the current
TeXmacs encoding and unicode and Html. Would you be willing to help on this?

> 2) When and how will TeXmacs encoding englobe unicode? More concretely:
> I am trying to write an export filter from texmacs to xml, and I need a
> facility to convert texmacs strings to unicode strings. Should I try to
> implement a cork<->unicode conversion by myself, or would it be more
> sensible to wait?

We should implement a more general conversion scheme based on dictionaries.
Andrey's dictool can also deal with dictionaries for encodings.
We first should build a database for this and implement algorithms for
conversions based on this database.

Andrey: could you add an option to save an encoding dictionary
by ordering it on the second entries instead of the first.
This is more natural in the case of font layouts.

> > > BTW, has my string_search_and_replace.patch been or applied or has it
> > > been rejected? Just curious...
> > 
> > I don't remember having seen that patch (but I receive soooo many 
> > messages)...
> > If it is on Savannah, then I will take a look at it and let you know
> > as soon as I work through the incoming patches.
> 
> I originally posted the patch to texmacs-dev. I put it on Savannah just
> now.

Thanks; I just looked at it. I am not sure what you want to do exactly
with the new routines, but I will include your patch as soon as there
is a sensible application for them.