bug#17412: 24.3; Unicode key events broken, not usable in input method

[Stefan Dorn]

My keyboard layout includes Unicode keys like "ł", U+0142 (l with
stroke), and combining diacritics (U+0300 etc). I've been trying to use
them in quail layouts, eg:

  (quail-define-package
  "custom" "custom layout" "^" t
  "Proof-of-concept layout." nil t t nil nil nil nil nil nil nil t)

  (quail-define-rules ("ł" ?l))

The key is never passed into the input-method-function, and so just
inserted literally. (Typing Unicode keys directly works fine.)

Digging around in keyboard.c, I found that read_char() only passes
events with keycode < 256 (line 3050ff) to input-method-function:

  /* Pass this to the input method, if appropriate.  */
  if (INTEGERP (c)
      && ! NILP (Vinput_method_function)
      /* Don't run the input method within a key sequence,
     after the first event of the key sequence.  */
      && NILP (prev_event)
      && ' ' <= XINT (c) && XINT (c) < 256 && XINT (c) != 127)

Using read-key-sequence, Emacs seems to parse "ł" as [322] (0x142 in
decimal). Disabling the condition in read_char() (so the key is
actually passed to quail) only seems to cause an infinite loop in quail
that I've not been able diagnose yet.

[322] as key event seems strange to me. The XLib keycode for "ł" (as
reported by xev) is 0x1000142. Maybe Emacs cuts off the leading bit?

Interestingly, quail shows the key in the guidance screen just fine, ie:

  (quail-define-rules ("xł" ?l))

and typing "x" correctly suggest "xł" as a pattern; it's just impossible
to pass "ł" to quail and have it be parsed correctly.


In GNU Emacs 24.3.1 (x86_64-pc-linux-gnu, X toolkit)
 of 2014-05-05 on scabeiathrax
Windowing system distributor `The X.Org Foundation', version 11.0.11500000
System Description:    NAME=Gentoo

Configured using:
 `configure '--prefix=/usr' '--build=x86_64-pc-linux-gnu'
 '--host=x86_64-pc-linux-gnu' '--mandir=/usr/share/man'
 '--infodir=/usr/share/info' '--datadir=/usr/share' '--sysconfdir=/etc'
 '--localstatedir=/var/lib' '--libdir=/usr/lib64'
 '--disable-silent-rules' '--disable-dependency-tracking'
 '--program-suffix=-emacs-24' '--infodir=/usr/share/info/emacs-24'
 '--localstatedir=/var'
 '--enable-locallisppath=/etc/emacs:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/gcc/x86_64-pc-linux-gnu/4.8.2/../../../../lib64'
 '--with-gameuser=games' '--without-compress-info' '--without-hesiod'
 '--without-kerberos' '--without-kerberos5' '--with-gpm' '--with-dbus'
 '--without-gnutls' '--without-xml2' '--without-selinux'
 '--without-wide-int' '--with-sound' '--with-x' '--without-ns'
 '--without-gconf' '--without-gsettings' '--without-toolkit-scroll-bars'
 '--with-gif' '--with-jpeg' '--with-png' '--with-rsvg' '--with-tiff'
 '--with-xpm' '--without-imagemagick' '--with-xft' '--with-libotf'
 '--with-m17n-flt' '--with-x-toolkit=lucid' '--with-xaw3d'
 'GENTOO_PACKAGE=app-editors/emacs-24.3-r4'
 'build_alias=x86_64-pc-linux-gnu' 'host_alias=x86_64-pc-linux-gnu'
 'CFLAGS=-O2 -pipe -march=core2' 'LDFLAGS=-Wl,-O1 -Wl,--sort-common
 -Wl,--hash-style=gnu -Wl,--as-needed' 'CPPFLAGS=''

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
( r e d a <backspace> <backspace> a d - k e y - s e
q u e n c e SPC " k e y : SPC " ) <left> <left> C-M-x
̈ C-M-x ł C-M-x e M-x b u g <tab> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> <backspace> <backspace> <backspace>
<backspace> <backspace> r e p o <tab> r t <tab> <r
eturn>

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
[776]
[322]
"e"
Making completion list...

Load-path shadows:
None found.

Features:
(shadow sort gnus-util mail-extr emacsbug message format-spec rfc822 mml
mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231 mailabbrev
gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils help-mode easymenu time-date tooltip ediff-hook
vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind dynamic-setting
font-render-setting x-toolkit x multi-tty emacs)