Re: [nmh-workers] mhshow: invalid BASE64 encoding in --

[Laura Creighton]

With  irony, in a discussion about how to handle bad encodings in mail, I
found that I could not read this message by Valdis Klētnieks
https://lists.nongnu.org/archive/html/nmh-workers/2019-03/msg00023.html

Something bad seems to have happened to his encoding of a '§'.

Now, my .mh_profile says (all one line, but I made it more readable).

    mhshow-show-text: iconv -f "$(charset=$(echo %a | 
    sed -n -r 's/.*charset="?([-a-zA-Z0-9_]*).*/\1/p');
    if [ x$charset = xunicode-1-1-utf-7 ]; then echo utf-7;
    else echo ${charset:-iso-8859-1}; fi)" | less

(I used to get lots of utf-7 mail.  Haven't seen any for man years now.)

and iconv is very picky.  It says, quite correctly, 
iconv: illegal input sequence at position 507 and then stops.

So my experience is seeing:
>Date:    Sun, 17 Mar 2019 18:12:49 -0400
>To:      Ken Hornstein <address@hidden>
>cc:      address@hidden
>From:    "Valdis Klētnieks" <address@hidden>
>Subject: Re: [nmh-workers] mhshow: invalid BASE64 encoding in --
>
>
>iconv: illegal input sequence at position 507On Sun, 17 Mar 2019 17:29:16 
>-0400, Ken Hornstein said:
>
>> >My reading of RFC2045 says a conforming base64 decoder is allowed to toss 
>> >out
>> >the blanks and the '!' char and decode the rest.
>> >
>> >   Any characters outside of the base64 alphabet are to be ignored in
>> >   base64-encoded data.
>> >
>> >Yeah.  That's pretty definitive. :)
>>
>> Oh, hm, you know you learn something new every day, and this is my new
>> thing for today.  As much as I've read RFC 2045 over the years, I missed
>> this!  (This is in -- 
>nmh-workers
>https://lists.nongnu.org/mailman/listinfo/nmh-workers

I can see how this might be behaviour you might want, but mostly I don't.


You can give iconv the -c flag "Silently discard characters that cannot be
                                converted instead  of terminating when
                                encountering such characters."

But since it is silent, there is no way for me to know that it encountered a
problem.  

But the behaviour I want is one that iconv doesn't give you.
Scream informatively about the problem and then continue on as if it 
never happened.  I want the message that I get without the -c flag and then
the -c behaviour.

When my mail blows up, I just pop into .mh_profile, add the -c flag, and then
find out what it was that Valdis wanted to tell us.  Then I take it out again
so I can be informed when iconv next runs into problems.

I just thought that while we were discussing what we should do, I would
mention this because it is the middle ground that I want most of the time.

Laura