Re: [Nmh-workers] Weird behavior with non-ascii code in headers

[Ken Hornstein]

>I think this ties in with my Yahoo Groups email header parsing problem
>where scan can't deal with incorrectly formed header lines.

Different layers; your problem was with the input routines (header parsing).
Valdis's problem was with the output routines (specifically, format library).
The headers of his message were parsed fine.

>1) Make command processing more robust.
>2) Validate header upon incorporation.

I'm all for that ... but it just brings up some sticky issues.  Like if
you have a header which is not valid, then what, exactly, should you
do about it?

>I think it would be great to make all commands a bit more robust when
>dealing with errors like this, but it seems to me that having some kind
>of header validation/correction at the beginning of the process would
>help avoid these problems.  I took a quick look at inc.c, and it appears
>someone was already working on massaging the "From" line, but that code
>block is commented out.

I'm looking at inc.c and I'm not seeing the code you mention; can you point
me to it?

>Should we have a standalone header validation program that is only run
>for cases like this or should it be some kind of subroutine that can be
>called by other commands?

David Levine already wrote mhfixmsg; it occurs to me that this might be
a good candidate for that.

>P.S. this will show my lack of familiarity with nmh code (so far), but
>do we care about RFC 5335 compliance at all?

I have not yet seen a message/global MIME type in the wild.  When we start
seeing them I think we should care.  Are people seeing these messages?
It looks like RFC 5335 was superceded by RFC 6532.  The other part of this
is RFC 6531, which defines SMTPUTF8.  A quick perusal of larger email
providers (gmail, yahoo) does NOT show support for SMTPUTF8.  And I'm
not really looking forward to implementing RFC 5504.  So I think we
can safely punt on this for now.  I think the changes should be minimal,
though ... we just convert everything from UTF-8 to the native character
set during the output routines (this only applies to RFC 6532).

--Ken