Re: Releasing the thread global_lock from the module API

[sbaugh]

Eli Zaretskii <eliz@gnu.org> writes:
>> From: Spencer Baugh <sbaugh@janestreet.com>
>> Cc: emacs-devel@gnu.org
>> Date: Fri, 01 Mar 2024 16:56:55 -0500
>> 
>> Eli Zaretskii <eliz@gnu.org> writes:
>> 
>> >> Then I would release the lock and call into my library, which does some
>> >> useful work which takes a while.
>> >
>> > How is this different from starting your own native thread, then
>> > releasing the lock?
>> 
>> They are completely different things.
>
> I think there's a misunderstanding here, because I cannot see how they
> could be "completely different things."  See below.

Oh, yes, you are correct, I didn't understand what you were proposing.
Thank you for your patience, your longer explanation has made your
proposal clear to me now.

>> > If a C library provides some blocking function which does some
>> > complicated form of IO, a module providing bindings for that C library
>> > can release global_lock before calling that function, and then
>> > re-acquire the lock after the function returns.  Then Lisp threads
>> > calling this module function will not block the main Emacs thread.
>> 
>> So, if I am running the following program in a Lisp thread:
>> 
>> (while (do-emacs-things)
>>   (let ((input (do-emacs-things)))
>>     (let ((result (call-into-native-module input)))
>>       (do-emacs-things result))))
>> 
>> and call-into-native-module unlocks the global lock around the calls
>> into my library, then during the part of call-into-native-module which
>> calls into my library, the main Emacs thread will not be blocked and can
>> run in parallel with call-into-native-module.
>
> I'm saying that your call-into-native-module method could do the
> following:
>
>   . start a native thread running the call into your library
>   . release the global lock by calling thread-yield or sleep-for or
>     any other API which yields to other Lisp threads
>   . wait for the native thread to finish
>   . return when it succeeds to acquire the global lock following the
>     completion of the native thread
>
> The _only_ difference between the above and what you described is that
> portions of call-into-native-module are run in a separate native
> thread, and the Lisp thread which runs the above snippet keeps
> yielding until the native thread finishes its job.  How is this
> "completely different"?

Oh, yes, that is similar to what I'm proposing.

Just to summarize, if call_into_native_module looks like:

emacs_value call_into_native_module(emacs_env *env, emacs_value input) {
  native_value native_input = convert_to_native(env, input);
  native_value native_output;

  [...some code...]

  return convert_to_emacs(env, native_output);
}

Then the current state of the world ("hold the lock" model):

  native_output = some_native_function(native_input);

If I understand correctly, you are proposing (the "new thread" model):

  native_thread_handle handle = native_thread_create(some_native_function, 
native_input);
  while (!native_thread_done(handle)) {
    emacs_thread_yield(env);
  }
  native_output = native_thread_result(handle);

And I am proposing (the "release lock" model):

  release_global_lock(env);
  native_output = some_native_function(native_input);
  acquire_global_lock(env);
  
All three of these are used in the same way from Lisp programs.  But the
"new thread" and "release lock" models have the advantage over the "hold
the lock" model that if called from a Lisp thread, that Lisp thread will
run some_native_function in parallel with Lisp execution on other Lisp
threads, including the main Emacs thread.

To check my understanding: does this all seem correct so far, and match
your proposal?

So, the main difference between the "new thread" model and the "release
lock" model is that creating a native thread takes a nontrivial amount
of time; maybe around 0.1 milliseconds.  If some_native_function would
takes less time than that, the thread creation cost will slow down
Emacs, especially because the native module creates the native thread
while holding the Lisp global_lock.

Reducing the thread creation cost is quite hard; a thread pool can help,
but that is complex and still has substantial cost in communication and
context switching.

So, to avoid the thread creation overhead, a native module should only
create a native thread to run some_native_function if the native module
knows that some_native_function will take a long time.

Unfortunately, most of the time a native module can't know how long
some_native_function will take before calling it.  For example, native
functions which make network calls might return immediately when there's
already data available from the network, but will have to send new
network requests sometimes.  Or, native functions which provide an
in-memory database might be fast or slow depending on the database
contents.

Since the native module doesn't know if some_native_function will take a
long time, the native module needs a way to allow some_native_function
to run in parallel which is cheap, so the native module can do it for
all calls to some_native_function.

The "release lock" model fits this need.  Releasing the lock is
essentially free, and allows some_native_function to run in parallel
with other Lisp threads.  Re-acquiring the lock afterwards will take
time if the operating system switched to other Lisp threads which
acquired the lock, but that's OK: the other Lisp threads are doing
useful work in Lisp in parallel with call_into_native_module waiting for
them to release the lock.