bug#1174: 23.0.60; Some UTF-8 mails displaying wrongly in Emacs 23

[Reiner Steib]

On Sat, Nov 29 2008, Stefan Monnier wrote:

>> Stefan, what are the "accidental emacs-mule-valid byte sequences"
>> that trigger this problem?  It would be good if someone could send
>> me a problematic article. [2]
>
> In Emacs-22, the problem was more difficult to trigger: you had to
> receive an email whose undecoded text contained emacs-mule escape
> sequences, which is rather uncommon.  With Emacs-23, it's a lot more
> common since the internal encoding has changed to a variant of utf-8:
> an 8bit body using utf-8 will see its content unwillingly decoded during
> nnimap-demule which leads to the bugs we've seen recently.

Could you send me an article demonstrating the problem?

> I'm pretty sure that string-as-multibyte is wrong here in general.
> Maybe the problem is that nnimap-demule is used blindly in different
> contexts where some need string-to-multibyte and some need
> string-as-multibyte.  E.g. maybe Simon's problem was linked to imap
> groups with non-ASCII chars in their names, rather than in the
> message bodies.

I'm not familiar with the IMAP code, but AFAICS, `nnimap-demule' is
only used when getting headers or body:

| nnimap.el:611:              headers (nnimap-demule

(defun nnimap-retrieve-headers-progress ()
  "Hook to insert NOV line for current article into `nntp-server-buffer'."

| nnimap.el:951:         (nnimap-demule

(defun nnimap-callback (article gnus-callback buffer)
  (when (eq article (imap-current-message))
    (remove-hook 'imap-fetch-data-hook
                 (nnimap-make-callback article gnus-callback buffer))
    (with-current-buffer buffer
      (insert
       (with-current-buffer nnimap-server-buffer
         (nnimap-demule
          (if (imap-capability 'IMAP4rev1)
              ;; xxx don't just use car? alist doesn't contain
              ;; anything else now, but it might...
              (nth 2 (car (imap-message-get article 'BODYDETAIL)))
            (imap-message-get article 'RFC822)))))
      (nnheader-ms-strip-cr)
      (funcall gnus-callback t))))

| nnimap.el:977:                (insert (nnimap-demule (if detail

(defun nnimap-request-article-part (article part prop &optional
                                            group server to-buffer detail)

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/