Re: [Texmacs-dev] character encoding

[Joris van der Hoeven]

Hi Felix,

> Good news! I am done with my work on character conversion. I submitted
> the patch on savannah, but my page carries a bit of extra information.
> See:
> 
> http://www.fbreuer.de/texmacs

Great; thanks a lot. I have integrated your code into the next
version of TeXmacs. However, some final cleanup is necessary:

1) Inside the TeXmacs code, you should (almost) never use '.'
   for data access, but rather '->'. The reason is that '->' can be
   customized while '.' cannot. Therefore, you should place methods
   like 'add_child', 'set_label', etc. in the representation class.

2) Converters should be resources (like dictionaries).
   This is the standard mechanism for guaranteeing that the conversion
   tables will be loaded only once.

3) Maybe you should replace 'tree' by 'ht' when using it as a variable
   in order to avoid confusion with the 'tree' class.

4) David should create a new macro CONCRETE_TEMPLATE_2_VAR or
   CONCRETE_NULL_TEMPLATE_2_VAR for dealing with the required
   memory management. This is useful in order to keep such
   non trivial routines centralized.

I propose the following working scheme: I first release TeXmacs 1.0.1.2,
Felix takes care of 1, 2 and 3 and writes a patch which will be integrated
in version 1.0.1.3. Then David takes care of 4 and the corresponding patch
will be integrated in version 1.0.1.4.

In the meantime, I added routines utf8->cork and cork->utf8 to
the scheme interface; you can add more if you want.

I hope that <iconv.h> is sufficiently standard so that
we do not get any compilation problems on exotic platforms.
Otherwise, the configure.in will need to be modified.
We might also automatically generate tables using <iconv.h>
and include them into TeXmacs. This latter strategy would allow
us to perform operations on the dictionaries as a whole.

It would be good to include tables for the existing converters
        string iso_to_koi8 (string s);
        string koi8_to_iso (string s);
        string iso_to_koi8uk (string s);
        string koi8uk_to_iso (string s);
        string il2_to_cork (string s);
        string cork_to_il2 (string s);
from analyze.hh, so as to replace the current conversion scheme
by the more uniform scheme by Felix. After that, we should also
take care of the .enc files.

Once more many thanks to Felix,

Joris