Re: sort-coding-systems does not prefer UTF-8 over UTF-16 (and mac-roman)

[Kenichi Handa]

In article <address@hidden>, Jesper Harder <address@hidden> writes:
> With the recent introduction of the new coding systems:

>      mule-utf-16
>      mule-utf-16-be-with-signature
>      mule-utf-16-le-with-signature

> `sort-coding-systems' no longer prefers UTF-8.

> I think the new coding systems need to be added to the clause related to
> UTF-16 in `sort-coding-systems'.

Thank you for pointing it out.  I've just installed the
attached fix.  In it, I directly lowered their priorities.

> Also, UTF-8 is not preferred over mac-roman.  I think UTF-8 should
> always have higher priority, since mac-roman is a highly platform
> specific encoding.

I put higher priority to a coding system listed in
coding-category-list.  So, unless mac-roman is explicitely
preferred, utf-8 should have higher priority now.

---
Ken'ichi HANDA
address@hidden

2003-04-11  Kenichi Handa  <address@hidden>

        * international/mule-cmds.el (sort-coding-systems): Set lower
        priority to a utf-16 base coding system.  Set higher priority to a
        coding system listed in coding-category-list.

Index: mule-cmds.el
===================================================================
RCS file: /cvsroot/emacs/emacs/lisp/international/mule-cmds.el,v
retrieving revision 1.226
retrieving revision 1.228
diff -u -c -r1.226 -r1.228
cvs server: conflicting specifications of output style
*** mule-cmds.el        18 Mar 2003 04:23:50 -0000      1.226
--- mule-cmds.el        11 Apr 2003 01:20:36 -0000      1.228
***************
*** 416,458 ****
  non-nil, it is used to sort CODINGS in the different way than above."
    (if sort-coding-systems-predicate
        (sort codings sort-coding-systems-predicate)
!     (let* ((most-preferred (symbol-value (car coding-category-list)))
           (lang-preferred (get-language-info current-language-environment
                                              'coding-system))
           (func (function
                  (lambda (x)
                    (let ((base (coding-system-base x)))
!                     (+ (if (eq base most-preferred) 64 0)
!                        (let ((mime (coding-system-get base 'mime-charset)))
                           ;; Prefer coding systems corresponding to a
                           ;; MIME charset.
                           (if mime
                               ;; Lower utf-16 priority so that we
                               ;; normally prefer utf-8 to it, and put
                               ;; x-ctext below that.
!                              (cond ((or (eq base 'mule-utf-16-le)
!                                         (eq base 'mule-utf-16-be))
!                                     16)
                                     ((string-match "^x-" (symbol-name mime))
!                                     8)
!                                    (t 32))
                             0))
!                        (if (memq base lang-preferred) 8 0)
!                        (if (string-match "-with-esc\\'" (symbol-name base))
!                            0 4)
!                        (if (eq (coding-system-type base) 2)
!                            ;; For ISO based coding systems, prefer
!                            ;; one that doesn't use escape sequences.
!                            (let ((flags (coding-system-flags base)))
!                              (if (or (consp (aref flags 0))
!                                      (consp (aref flags 1))
!                                      (consp (aref flags 2))
!                                      (consp (aref flags 3)))
!                                  (if (or (aref flags 8) (aref flags 9))
!                                      0
!                                    1)
!                                2))
!                          1)))))))
        (sort codings (function (lambda (x y)
                                (> (funcall func x) (funcall func y))))))))
  
--- 416,473 ----
  non-nil, it is used to sort CODINGS in the different way than above."
    (if sort-coding-systems-predicate
        (sort codings sort-coding-systems-predicate)
!     (let* ((from-categories (mapcar #'(lambda (x) (symbol-value x))
!                                   coding-category-list))
!          (most-preferred (car from-categories))
           (lang-preferred (get-language-info current-language-environment
                                              'coding-system))
           (func (function
                  (lambda (x)
                    (let ((base (coding-system-base x)))
!                     ;; We calculate the priority number 0..255 by
!                     ;; using the 8 bits PMMLCEII as this:
!                     ;; P: 1 iff most preferred.
!                     ;; MM: greater than 0 iff mime-charset.
!                     ;; L: 1 iff one of the current lang. env.'s codings.
!                     ;; C: 1 iff one of codings listed in the category list.
!                     ;; E: 1 iff not XXX-with-esc
!                     ;; II: if iso-2022 based, 0..3, else 1.
!                     (logior
!                      (lsh (if (eq base most-preferred) 1 0) 7)
!                      (lsh
!                       (let ((mime (coding-system-get base 'mime-charset)))
                           ;; Prefer coding systems corresponding to a
                           ;; MIME charset.
                           (if mime
                               ;; Lower utf-16 priority so that we
                               ;; normally prefer utf-8 to it, and put
                               ;; x-ctext below that.
!                              (cond ((string-match "utf-16"
!                                                   (symbol-name mime))
!                                     2)
                                     ((string-match "^x-" (symbol-name mime))
!                                     1)
!                                    (t 3))
                             0))
!                       5)
!                      (lsh (if (memq base lang-preferred) 1 0) 4)
!                      (lsh (if (memq base from-categories) 1 0) 3)
!                      (lsh (if (string-match "-with-esc\\'"
!                                             (symbol-name base))
!                               0 1) 2)
!                      (if (eq (coding-system-type base) 2)
!                          ;; For ISO based coding systems, prefer
!                          ;; one that doesn't use escape sequences.
!                          (let ((flags (coding-system-flags base)))
!                            (if (or (consp (aref flags 0))
!                                    (consp (aref flags 1))
!                                    (consp (aref flags 2))
!                                    (consp (aref flags 3)))
!                                (if (or (aref flags 8) (aref flags 9))
!                                    0
!                                  1)
!                              2))
!                        1)))))))
        (sort codings (function (lambda (x y)
                                (> (funcall func x) (funcall func y))))))))