Re: [Nmh-workers] 128 byte field name limit in NAMESZ breaks scan(1)

[Jeffrey C Honig]

If I read RFC0822 correctly field-name does not have a length limit,
it's one or more characters.  Given that, using a fixed value of NAMESZ
in m_getfld() is an invalid assumption.

The proper fix would involve a dynamically allocated buffer and an
examination of all uses of m_getfld() to update it.

A quick look shows that most of the calls to m_getfld() define name char
name[NAMSZ] and pass it to m_getfld().  If they don't assume a length, a
quick change to m_getfld() to return a pointer to a buffer and
reallocate it as necessary should only take a hour or two.  The hardest
part would be making sure that there are not additional calls to
m_getfld() while the value in the buffer returned from the first call
are expected to remain valid.

Thanks (谢谢).

Jeff

address@hidden wrote:

> 
> 
> In the message dated: Wed, 22 Oct 2008 09:00:12 BST,
> The pithy ruminations from Peter Maydell on 
> <Re: [Nmh-workers] 128 byte field name limit in NAMESZ breaks scan(1)> were:
> => David Levine wrote:
> => >Mark wrote:
> => >> Peter Maydell wrote:
> => >> => I think this has been reported on the bug tracking system before.
> => >> => (https://savannah.nongnu.org/bugs/?14975)
> => >> 
> => >> Yes, that does sound like the issue I'm seeing. Note that the bug
> => >> was opened in 2005. I'm really not a "C" programmer, but it took me
> => >> almost as long to read the bug report as it did to increase the
> => >> value of NAMESZ. As far as I'm concerned, this is a trivial fix to a
> => >> signficant interface issue (ie., not being able to see the fields in
> => >> the scan listing).
> => 
> => Yes, but I'm not sure it's the right fix. Most of that bug is me trying
> 
> Philosophically, it may not be the "right" fix. Practically, it fixes the 
> problem with scan.
> 
> Perhaps the "right" fix is to simply ignore lines in a mail file that preceed 
> the body text but are not valid headers. As per RFC 1122:
> 
>       "Be liberal in what you accept, and conservative in what you send." 
> 
> This seems to be the current behavior, as long as the invalid line is less 
> than 
> 128 characters.
> 
> When scan encounters a "From " line (note the missing colon), scan is happy to
> accept it, unless it's over 127 characters...in which case the complaint is
> about the length, implying that there's nothing wrong with a "From " line in 
> the
> header block.
> 
>       
> => to get information from the guy who reported it. (It didn't help that the
> => file he attached as a test case had got mangled in transit somehow so that
> => it started not "From ..." but ">From ...".)
> => 
> => >> => in a mail spool file and so get stripped out as the messages are
> => >> => split). So perhaps this bug should be fixed by making whatever
> => >> => path you're using to put messages in the folder strip out the
> => >> => envelope From as inc does.
> => >
> => >Ah, I had procmail configured to insert the 'From ' line,
> => >with its -f option.
> => 
> => Well, er, don't do that then.
> => 
> => The mh-mail(5) manpage is pretty clear about what the format of files
> => in an nmh mail directory ought to be. These 'From ' lines simply aren't
> 
> Actually, the man page states:
> 
>       It should be noted that although neither Bell nor Berkeley mailers
>       produce message files in the format that  nmh  prefers,  nmh  can
>       read message files in that antiquated format.
> 
> That "antiquated format", as far as I can tell, is the mbox specification, 
> where messages begin with a "From " line. Thus, my reading of the mh-mail 
> manpage is that nmh doesn't necessarily use the "From " line, doesn't write a 
> "From " line, but that it should accept a "From " line. In fact, it _does_ 
> accept a "From " line (thus, there's no need to try to track down every 
> program 
> in the universe that creates or doesn't remove an existing "From " line and 
> provide instructions for how to make them compatible with nmh). The problem 
> seems to be that nmh just doesn't handle "From " lines (and, I suspect, other 
> headers) longer than 128 characters.
> 
> 
> => valid within it.
> => 
> => If the code writing 'From ' lines is part of nmh we should fix it;
> => if it's third-party then we should probably identify and document
> => how to configure that code to write valid nmh files.
> => 
> => >> Perhaps, as the bug report from 2005 stated, that would break mail
> => >> filtering that actually looks at the "From " line. Why strip it
> => >> out...it's data about the mail delivery process, and may have some
> => >> value to some other process (spam filtering? mail statistics? random
> => >> number generation?).
> => 
> => If you want the information you should write it in as an X-Envelope-From:
> => header or similar.
> 
> Maybe. Some programs (anything that deals with the mbox format, for example) 
> expect "From ", not "X-Envelope-From:".
> 
> => 
> => >> Personally, I think that the philosophical argument about whether
> => >> tools strip out the "From " line or not is really separate from the
> => >> fact that the the scan listing breaks because of an arbitrarily
> => >> small value in NAMESZ. The first issue is difficult to resolve (as
> => >> evidenced by the 3 year-old bug), while the second issue is trivial
> => >> to fix (though maybe not as "pure" as some would want)....but what
> => >> do I know? :)
> => >
> => >I agree.  Any objection to raising NAMESZ to 512?
> => 
> => It's still an arbitrary limit. I'd be more interested in raising
> => it if there were actually problems because of messages with header
> => components with names longer than 128. (The 'From ' lines don't
> => count because they're not valid syntax.)
> 
> Again, the mh-mail manpage states that nmh will read messages that contain 
> those lines, even if they aren't used by nmh, so they are not defacto invalid 
> syntax to nmh.
> 
> => 
> => (If you do raise it then my reading of RFC2822 says 998 would
> => be more defensible than 512.)
> 
> Perhaps the mh-mail man page needs to be updated. My mh-mail manpage, from the
> 1.3 source, cites a limit of 63 characters, as per RFC822.
> 
> 
> In any case, raising the limit does seem to be the answer...
> 
> 
> Mark "feeling pedantic this morning" Bergman
> 
> => 
> => -- PMM
> => 
> 
> 
> 
> 
> 
> _______________________________________________
> Nmh-workers mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/nmh-workers
> 
> 

-- 
Jeffrey C. Honig <address@hidden>
http://www.honig.net/jch
GnuPG ID:14E29E13 <http://www.honig.net/jch/key.shtml>