Re: Unibyte characters, strings, and buffers

[David Kastrup]

"Stephen J. Turnbull" <address@hidden> writes:

> Andreas Schwab writes:
>  > "Stephen J. Turnbull" <address@hidden> writes:
>  > 
>  > > *sigh*  No, it's about unibyte being a premature pessimization.
>  > 
>  > Unibyte is a pure space optimisation.
>
> It may be a space optimization, but it's hardly pure.  Else this
> discussion wouldn't be happening.  And `string-as-unibyte' exposes the
> internal representation of strings to Lisp.
>
>  > Everything else should work as if all bytes in the range 128-255
>  > are decoded in the eight-bit charset.
>
> There seem to be conflicting opinions about that, and I would
> certainly disagree as there are scads of European charsets that
> happily fit into bytes.

That's not what unibyte buffers are for.  They are for byte streams, not
characters.  You would not want to edit a unibyte buffer, for example,
by inserting text and stuff.

Now for byte stream manipulation, code points other than 0..255 are a
nuisance.  Certainly a larger nuisance than having to clear
case-fold-search if you really want to do a byte search.

> I see no reason why character operations (such as case conversion)
> shouldn't work transparently on bytes in GR interpreted as the
> corresponding Latin-1 (or any ISO Latin) charset -- with a little
> extra metadata in (internal unibyte) buffers and strings to indicate
> the charset implied.  (This charset is independent of the various
> coding systems associated with buffers; it only says how to interpret
> a byte as a character in operations on characters in buffers.)

We have that "extra metadata", it is the unibyte flag.  But I consider
it a mistake to use it for anything but "character codes in this buffer
happen to range from 0..255 rather than 0..1000000 or whatever".

And since Unicode 128..255 happens to be the latin-1 plane where the
latin-1 plane is defined as all, this will mean that the result will
behave like the latin-1 plane.

Exactly because Emacs has _one_ underlying character set which happens
to be Unicode.

Which does not mean that it would be a good idea to use unibyte
buffers/strings for actual text that happens to be Latin-1 only.

-- 
David Kastrup