bug#18222: 24.3.92; fork handlers in gmalloc.c can lead to deadlock

[Ken Brown]

On 8/9/2014 3:24 PM, Ken Brown wrote:
> On 8/9/2014 3:44 AM, YAMAMOTO Mitsuharu wrote:
>>>>>>> On Fri, 08 Aug 2014 09:09:31 -0400, Ken Brown 
>>>>>>> <kbrown@cornell.edu> said:
>>
>>> malloc_enable_thread() in gmalloc.c calls pthread_atfork to set up fork
>>> handlers.  There are a couple of problems with this, but the immediate
>>> reason for this bug report is a problem on Cygwin that was reported in
>>> the thread starting at
>>
>>>     https://cygwin.com/ml/cygwin/2014-07/msg00387.html
>>
>>> and continuing at
>>
>>>     https://cygwin.com/ml/cygwin/2014-08/msg00001.html.
>>
>>> The issue is that the 'prepare' fork handler locks the pthread_mutexes
>>> prior to forking, and the ensuing processing of the fork command by the
>>> Cygwin DLL leads to a call to malloc in the same thread, resulting in
>>> deadlock.  This is a long-standing problem, but it was masked until
>>> recently by the fact that pthread_mutexes on Cygwin were ERRORCHECK
>>> mutexes by default.  As of Cygwin 1.7.31, pthread_mutexes are now NORMAL
>>> mutexes by default, so the problem has shown up.
>>
>>> A simple short-term workaround would be to explicitly set the mutexes to
>>> be ERRORCHECK or RECURSIVE mutexes on Cygwin, thereby restoring the
>>> previous behavior.  But this does not seem like the right long-term
>>> solution, for the reasons explained here:
>>
>>>     https://cygwin.com/ml/cygwin/2014-08/msg00161.html
>>>     https://cygwin.com/ml/cygwin/2014-08/msg00175.html
>>
>>> I know nothing about this other than what I learned from the two
>>> messages above, so I would appreciate some guidance.
>>
>> Originally, gmalloc.c bundled with Emacs was not thread-safe, so I
>> added some mutex code as a short-term solution:
>>
>>    http://lists.gnu.org/archive/html/emacs-devel/2007-06/msg01782.html
>>
>> Thread-safe malloc was required mainly for GLib (via GTK+, for
>> example), and atfork handers were necessary because the threads
>> internally used by GLib were not under our control.
>>
>> All the platforms I'm currently working at use their system malloc
>> rather than Emacs's gmalloc.c, so I don't think I can be of much help
>> about this issue.  If changing mutex attributes works well, then I
>> think that would be good enough for a workaround for upcoming 24.4
>> release.
> 
> OK.  For maximum safety, I should probably set the type of the mutexes 
> to ERRORCHECK, as they were in Cygwin 1.7.30, even though I think 
> RECURSIVE would work just as well.

The patch below does this.  Stefan, is it OK to install this in the emacs-24 
branch?

Ken

=== modified file 'src/gmalloc.c'
--- src/gmalloc.c       2014-03-04 19:02:49 +0000
+++ src/gmalloc.c       2014-08-10 13:24:52 +0000
@@ -490,8 +490,18 @@
 }

 #ifdef USE_PTHREAD
+/* On Cygwin prior to 1.7.31, pthread_mutexes were ERRORCHECK mutexes
+   by default.  When the default changed to NORMAL in Cygwin-1.7.31,
+   deadlocks occurred (bug#18222).  As a temporary workaround, we
+   explicitly set the mutexes to be of ERRORCHECK type, restoring the
+   previous behavior.  */
+#ifdef CYGWIN
+pthread_mutex_t _malloc_mutex = PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
+pthread_mutex_t _aligned_blocks_mutex = 
PTHREAD_ERRORCHECK_MUTEX_INITIALIZER_NP;
+#else  /* not CYGWIN */
 pthread_mutex_t _malloc_mutex = PTHREAD_MUTEX_INITIALIZER;
 pthread_mutex_t _aligned_blocks_mutex = PTHREAD_MUTEX_INITIALIZER;
+#endif /* not CYGWIN */
 int _malloc_thread_enabled_p;

 static void
@@ -526,14 +536,23 @@
      initialized mutexes when they are used first.  To avoid such a
      situation, we initialize mutexes here while their use is
      disabled in malloc etc.  */
+#ifdef CYGWIN
+  /* Use ERRORCHECK mutexes; see comment above. */
+  pthread_mutexattr_t attr;
+  pthread_mutexattr_init (&attr);
+  pthread_mutexattr_settype (&attr, PTHREAD_MUTEX_ERRORCHECK);
+  pthread_mutex_init (&_malloc_mutex, &attr);
+  pthread_mutex_init (&_aligned_blocks_mutex, &attr);
+#else  /* not CYGWIN */
   pthread_mutex_init (&_malloc_mutex, NULL);
   pthread_mutex_init (&_aligned_blocks_mutex, NULL);
+#endif /* not CYGWIN */
   pthread_atfork (malloc_atfork_handler_prepare,
                  malloc_atfork_handler_parent,
                  malloc_atfork_handler_child);
   _malloc_thread_enabled_p = 1;
 }
-#endif
+#endif /* USE_PTHREAD */

 static void
 malloc_initialize_1 (void)