bug#25214: 26.0.50; Interacting with user from threads other than the primary

[Michael Albinus]

Eli Zaretskii <eliz@gnu.org> writes:

> Here's some fun with threads:
>
>  emacs -Q
>
> Evaluate:
>
>   (defun my-question ()
>     (let (use-dialog-box) ;; in case this is a GUI frame
>       (if (y-or-n-p "Yes or no? ")
>         (message "You said YES")
>       (message "You said NO"))))
>
> Now evaluate this:
>
>   (make-thread #'my-question "question")
>
> You again get the question asked in the minibuffer, but typing
> anything at the prompt causes Emacs to complain that whatever you
> typed is "undefined".  Your only fire escape is to close this session
> with some mouse gesture, or kill it.
>
> What happens here is that, by the time the new thread runs, the main
> (a.k.a. "primary") thread is already idle, i.e. it already called
> read_char, which called wait_reading_process_output, which called
> thread_select.  (In fact, it's _because_ the primary thread called
> thread_select that the new thread was allowed to run at all, since it
> has to acquire the global lock for that, and that is only possible
> when the running thread releases the lock inside thread_select.)  Now,
> when wait_reading_process_output calls thread_select, it prepares the
> mask for file descriptors it will wait for to become ready for
> reading, in this case there's only one descriptor: the keyboard
> descriptor.  So the primary thread is waiting for input from the
> keyboard.
>
> Now the new thread starts running and eventually calls y-or-n-p, which
> calls read_char, which calls wait_reading_process_output.  But when
> this call prepares the mask for the thread_select call, it skips the
> keyboard descriptor, because we only allow a single thread to wait on
> each descriptor, and the keyboard descriptor is already watched by the
> primary thread.  So the new thread ends up not waiting on the keyboard
> input descriptor.
>
> The thread_select call then switches back to the primary thread, and
> the primary thread receives the Y or N character you type.  But it
> doesn't know what to do with it, so it becomes confused.
>
> IOW, user interaction from non-primary threads is currently inherently
> unsafe.

IOW, user interaction from non-primary threads is currently inherently
impossible.

I've tried to understand the mechanism in process.c, and I summarize it
(mainly for my understanding, but also to give you a chance to correct me).

The fd mask is controlled by fd_callback_info, an array (indexed by
fd's) over the struct fd_callback_data. There are two relevant struct
fields: thread and waiting_thread. thread is set in Fset_process_thread
for the infd and outfd file descriptors of the process, it shall be NULL
in the example given above.

waiting_thread is always set to current_thread, in the
compute_input_wait_mask, compute_non_process_wait_mask,
compute_non_keyboard_wait_mask and compute_write_mask functions. The
condition is always, that waiting_thread is either NULL or
current_thread.

A file descriptor, be it for the keyboard or process related or
whatever, is only taken into account during setting an fd_mask, when
either both thread and waiting_thread are NULL, or when one of the
struct fields is set to a thread, being equal to the current
thread. IIUC, thead has precedence over waiting_thread, but this might
be an implementation detail only. So far, it is expected that at least
one of the fields is NULL.

In order to fix the problem of reading input in a non-primary thread, we
need a new function which request I/O control to the current thread. It
needs

* to request control for the keyboard. This could be indicated by a
  signal to the main thread, which is *not* an error signal but a
  special one, let's call it `thread-set-keyboard'.

* to handle this signal in `thread-handle-event', by calling a
  respective function (let's call it `thread--set-keyboard'). This
  function sets the struct event thread to the thread which has
  delivered by the `thread-set-keyboard' signal. Next time fd_mask is
  prepared, the keyboard fd would be taken into account for the
  requesting thread.

* to stop/recall the recent pselect for the main thread in order to free
  the keyboard fd. Don't know how to do this.

* to provide a mechanism which resets the thread struct field of the
  keyboard fd to NULL, in order to let the main thread use the
  keyboard. Here I have also no idea how to do this.

> And then, of course, there's the use case where two threads ask the
> user something via the minibuffer.

This scenario shall also be possible then.

> Thoughts?

Here I am :-)

Best regards, Michael.