[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: coding tags and utf-16
From: |
Werner LEMBERG |
Subject: |
Re: coding tags and utf-16 |
Date: |
Sat, 24 Dec 2005 00:43:29 +0100 (CET) |
> There is a serious problem with coding tags and utf-16 encodings of
> any flavour: Emacs simply can't recognize the tag. [...]
Surprisingly, I saw no response on the list which either means that my
mail hasn't come through, nobody is interested in this problem, or
that it is a non-issue.
In case it won't get fixed I suggest to add it to the TODO list,
together with a not in the emacs manual that coding tags don't work
with utf-16 encoding flavours.
Werner
> This is a non-trivial problem. Right now I'm working on a groff
> preprocessor which tries to handle this. I'm doing the following to
> find the tag in an encoding-independent way:
>
> . Check whether the file starts with the BOM (Byte Order Mark) --
> this is one of the following byte sequences:
>
> UTF-8: 0xEFBBBF
> UTF-16: 0xFEFF or 0xFFFE
>
> Skip it.
>
> . Ignore zero bytes while looking for the -*- coding: ... -*-
> stuff.
>
> This heuristic algorithm might not give correct results in all cases
> but it should be sufficiently reliable for normal use.