Re: [Nmh-workers] scan %{body} bug?

[Ken Hornstein]

>Since I started using mh-v, for about two months, I've been keeping a
>pretty close eye on making sure scan and mhshow work perfectly.  This
>week collected four msgs where the decoding %{body} is truncated after 6
>to 16 characters with v1.2 and v1.4!

Okay, Steve was nice enough to give me a test message, so I took a look at
the problem.

Here's the issue.  When scansbr.c:scan() calls m_getfld() (that bastard
again!) it has the following short-circuit test when m_getfld() returns the
BODY state:

           if (! outnum) {
                state = FILEEOF; /* stop now if scan cmd */
                goto finished;
           }

(For us, outnum is zero).

This means that as soon as we get any amount of data for the BODY
state, we stop scanning the message.  In the message Steve pointed
me to the cutoff point in the scan listing is at byte 4096.  Since
the code enters the BODY state without reading more (normally you
would keep calling m_getfld() until you get FILEEOF) you get only
what's in the buffer; hence the truncation.  If the message headers
were shorter or longer, you'd never run into it.  I think this bug
has been around forever, and that 30 years ago no one had any idea
message headers would ever approach 4K.

So, how do we solve it?  Well, I'm not sure of the "right" thing to do.  It's
hard with the current format system to determine how many characters the body
needs to be.  We could take a guess.  If we guess wrong, that would result
in one extra read in some cases.  We know what "width" is, and the body can't
be longer than that.  So we could say that the body has to be at least
width characters (or we reach EOF).

Thoughts?

--Ken