Re: Worrying development

[Tom Lord]

    > From: Marius Vollmer <address@hidden>

    > > Well, heck.  In that case, maybe consider what I'm planning for Pika
    > > (at least initially).  Purely ASCII strings are stored 1-byte per
    > > character.  Most other strings 2-bytes per character.   Strings using
    > > characters outside the Basic Multilingual Plane, 4 bytes per
    > > character.

    > Yes, that's an attractive approach.  But I also find simply using
    > UTF-8 exclusively very attractive.  It might fit better with what
    > other people are doing and we might need fewer conversions when
    > wrapping external libraries.  Or maybe not.

In case it helps seduce you to the dark side of the force just a
little more:  

Having wrappings of external libraries mostly rely on
copying/converting strings is a win for thread support.   Having
FFI-using routines directly access or munge string data is, in
general, pretty touchy.   It is, I admit, a total pain in the butt
that so much existing code already does access string data directly --
but for the most part, that code is unlikely to be expecting UTF-8
anyway so......

-t