Re: [Nmh-workers] General question - unsupported charset conversion

[Ken Hornstein]

>My feeling is that if you don't recognize the source character set, you
>cannot possibly convert it to a display format in any secure manner.  By
>default I think we should not display the content, but instead spit out
>a diagnostic, with the option to re-run the show (or whatever) with a
>command-line option that passes the content through unconverted.

Context is important here: this came up in context of parsing MIME
parameters.  The case for dealing with text context is easier.  For
MIME parameters, I'm not sure it's as straightforward.

>I'm of mixed feelings about converting unknown characters to a proxy
>(e.g. '?'). This could be exploited to inject terminal escape sequences
>into xterm (or your VT220 – I know people who still use them!).

If we make sure we're converting all non-printable characters into something
else, I'm unclear as to how that could happen.  But if it can happen, please
educate me!  (For the record, my VT340 died a few years ago and it went to
electronics recycling).

>Yet another argument for declaring nmh a utf8-only zone, and convert
>everything to that on the way in.  We could bundle our own internal
>iconv and just call it a day.

I don't see how that really helps.  It would make programming easier,
somewhat (and presumably you'd just reject unknown character sets out of
hand during parsing).  We'd still have to deal with what happens when
you want to convert U+1F4A9 to ISO-8859-1.  I'm not saying it's not a bad
idea, though (although Aleksander definitely would not agree).

--Ken