[libmicrohttpd] "Failed to join a thread": Race condition when closing the connection?

[Jan-Benedict Glaw]

Hi!

We're using libmicrohttpd for serving API requests. That worked fine
until recently, when I changed popen()/pclose() to a more shallow (and
much faster) version (when the application VSZ is large.)

  It seems that I uncovered a small bug. I'm not yet all the way
through the code, but I guess that scheduling works different right
now and this leads to a race condition when running with one thread
per connection (MHD_USE_THREAD_PER_CONNECTION):

 daemon.c
~~~~~~~~~~
3024 MHD_cleanup_connections (struct MHD_Daemon *daemon)
3025 {
3026   struct MHD_Connection *pos;
3027 
3028   MHD_mutex_lock_chk_ (&daemon->cleanup_connection_mutex);
3029   while (NULL != (pos = daemon->cleanup_tail))
3030     {
3031       DLL_remove (daemon->cleanup_head,
3032                   daemon->cleanup_tail,
3033                   pos);
3034       MHD_mutex_unlock_chk_ (&daemon->cleanup_connection_mutex);
3035 
3036       if ( (0 != (daemon->options & MHD_USE_THREAD_PER_CONNECTION)) &&
3037            (! pos->thread_joined) &&
3038            (! MHD_join_thread_ (pos->pid.handle)) )
3039         MHD_PANIC (_("Failed to join a thread\n"));

Connections (and thus, threads) are popped off ->cleanup_tail one by
one and pthread_join() is called on them:

 mhd_threads.h
~~~~~~~~~~~~~~~
127 /**
128  * Wait until specified thread is ended and free thread handle on success.
129  * @param thread handle to watch
130  * @return nonzero on success, zero otherwise
131  */
132 #define MHD_join_thread_(thread) (!pthread_join((thread), NULL))

Testing it a bit, I found out that pthread_join() returns 3
(== ESRCH), which shall happen when "No thread with the ID thread
could be found." (man 3 pthread_join). I would have expected
"EINVAL thread is not a joinable thread." though...

  However, putting the pthread_join() or MHD_join_thread_() into a
small loop with a few msec delay in between fixed it, with an
occassional hint that it took a second try to reap the thread.

  So my guess is that in combination with one-thread-per-connection,
connection threads might be pthread_join()ed too fast, and only once
instead of actually waiting for them (as the comment might suggest.)

  Maybe somebody might want to have a closer look at
MHD_join_thread_() to actually make it wait? A simple (limited) loop
might just do the job.

Thanks,
  Jan-Benedict

-- 
Getslash GmbH, Hermann-Johenning-Platz 2, 59302 Oelde
Tel: +49-2522-834349-5    Fax:   +49-2522-834349-1
http://www.getslash.de    Mobil: +49-152-33822499
Sitz der Gesellschaft: Oelde
Handelsregister: Amtsgericht Münster, HRB 11911
Ust-Id-Nr.: DE 815060326
Geschäftsführung: Andre Peitz, Tobias Hanisch

signature.asc
Description: PGP signature