bug#3399: Crash in multi-TTY mode

[YAMAMOTO Mitsuharu]

>>>>> On 27 May 2009 02:44:23 -0000, "Shannon Jones" <cz2s20d02@sneakemail.com> 
>>>>> said:

> I'm almost embarrassed to report this, since it's rather strange and
> most likely unique to my setup.  Still, it involves a crash so I
> thought it would be worthwhile to see if anyone else can reproduce
> it.

I could reproduce it on GTK+/Mac OS X with the following steps.

  $ emacs -nw -Q
  M-x server-start RET
  $ emacsclient -c
  C-x 5 0 (on the X11 frame)

The problematic scenario is:

  1. XGetDefault is called in several GTK+ initialization steps.  It
     sets the XlibDisplayDfltRMDB bit in dpy->flags but dpy->db
     remains to be NULL in some cases (InitDefaults may return NULL).

    GetDflt.c (libX11-1.2.1):

    char *
    XGetDefault(
            Display *dpy,                       /* display for defaults.... */
            char _Xconst *prog,         /* name of program for option   */
            register _Xconst char *name)        /* name of option program wants 
*/
    {                                   /* to get, for example, "font"  */

        :

            /*
             * see if database has ever been initialized.  Lookups can be done
             * without locks held.
             */
            LockDisplay(dpy);
            if (dpy->db == NULL) {
                dpy->db = InitDefaults(dpy);
                dpy->flags |= XlibDisplayDfltRMDB;
            }
            UnlockDisplay(dpy);

        :

    }
  2. x_term_init calls XrmSetDatabase, but it doesn't reset the
     XlibDisplayDfltRMDB bit if display->db is NULL.  As a result,
     display->db holds a new database but XlibDisplayDfltRMDB is still
     set.

    Xrm.c (libX11-1.2.1):

    void XrmSetDatabase(
        Display *display,
        XrmDatabase database)
    {
        LockDisplay(display);
        /* destroy database if set up imlicitely by XGetDefault() */
        if (display->db && (display->flags & XlibDisplayDfltRMDB)) {
            XrmDestroyDatabase(display->db);
            display->flags &= ~XlibDisplayDfltRMDB;
        }
        display->db = database;
        UnlockDisplay(display);
    }

  3. x_delete_terminal does XrmSetDatabase(dpyinfo->display, NULL) to
     dissociate the database from the display.  Because
     XlibDisplayDfltRMDB is set, it destroys the database to be
     dissociated.

  4. x_delete_terminal then calls XrmSetDatabase and destroys the
     dissociated database again.  This causes crash.

I think this a bug in libX11.  It should either 1) not set
XlibDisplayDfltRMDB in XGetDefault unless dpy->db becomes non-NULL or
2) reset XlibDisplayDfltRMDB in XrmSetDatabase even if the previous
database is NULL.

Below is a possible workaround.  Unfortunately, we need to touch some
X11 internal data structures.

                                     YAMAMOTO Mitsuharu
                                mituharu@math.s.chiba-u.ac.jp

Index: src/xterm.c
===================================================================
RCS file: /sources/emacs/emacs/src/xterm.c,v
retrieving revision 1.1026
diff -c -p -r1.1026 xterm.c
*** src/xterm.c 19 May 2009 00:26:46 -0000      1.1026
--- src/xterm.c 27 May 2009 07:59:20 -0000
*************** along with GNU Emacs.  If not, see <http
*** 31,36 ****
--- 31,40 ----
  
  #ifdef HAVE_X_WINDOWS
  
+ /* This makes the fields of a Display accessible, in Xlib header files.  */
+ 
+ #define XLIB_ILLEGAL_ACCESS
+ 
  #include "lisp.h"
  #include "blockinput.h"
  
*************** x_term_init (display_name, xrm_option, r
*** 10285,10290 ****
--- 10289,10304 ----
  
    xrdb = x_load_resources (dpyinfo->display, xrm_option,
                           resource_name, EMACS_CLASS);
+   /* This is a workaround for a bug in some versions of libX11.
+      XGetDefault may set XlibDisplayDfltRMDB (1L << 7) in dpy->flags
+      (dpyinfo->display->private16) even when dpy->db
+      (dpyinfo->display->db) remains to be NULL, and XrmSetDatabase
+      doesn't clear the flag if dpy->db is NULL.  This causes crash in
+      the XrmDestroyDatabase call from x_delete_terminal because the
+      preceding XrmSetDatabase call destroys the associated database
+      because of the XlibDisplayDfltRMDB flag.  */
+   if (dpyinfo->display->db == NULL)
+     dpyinfo->display->private16 &= ~(1L << 7);
  #ifdef HAVE_XRMSETDATABASE
    XrmSetDatabase (dpyinfo->display, xrdb);
  #else