Re: Concurrency via isolated process/thread

[Eli Zaretskii]

> From: Ihor Radchenko <yantar92@posteo.net>
> Cc: luangruo@yahoo.com, emacs-devel@gnu.org
> Date: Mon, 24 Jul 2023 16:38:55 +0000
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> >> AFAIK, reading buffer does not require moving the gap.
> >
> > We've been through that: at least xml.c moves the gap to allow
> > external libraries access to buffer text as a single contiguous C
> > string.  This is a capability I wouldn't want to lose, because it
> > might come in handy in future developments.
> 
> move_gap_both should lock the buffer as it is buffer modification (of
> the buffer_text object). htmlReadMemory should also lock it because it
> expects constant string segment.

This basically means we will only allow access of a single thread to
the same buffer, see below.

> > I still don't quite see how this will work.  Indirect buffers don't
> > introduce parallelism, and programs that modify indirect buffers
> > _know_ that the text of the base buffer will also be modified.  By
> > contrast, a thread that has been preempted won't know and won't expect
> > that.  It could, for example, keep buffer positions in simple
> > variables, not in markers; almost all Lisp programs do that, and use
> > markers only in very special situations.
> 
> Any async thread should expect that current buffer might be modified.

That's impossible without rewriting a lot of code.  And even after
that, how is a thread supposed to "expect" such changes, when they can
happen at any point in the Lisp program execution?  What kind of
measures can a Lisp program take to 'expect" that?  The only one that
I could think of is to copy the entire buffer to another one, and work
on that.  (Which is also not fool-proof.)

> Or lock the buffer for text modifications explicitly (the feature we
> should probably provide - something more enforcing compared to
> read-only-mode we have now).

Locking while accessing a buffer would in practice mean only one
thread can access a given buffer at the same time.  Which is what I
suggested to begin with, but you said you didn't want to give up.

> > In addition, on the C level, some code computes pointers to buffer
> > text via BYTE_POS_ADDR, and then uses the pointer as any C program
> > would.  If such a thread is suspended, and some other thread modifies
> > buffer text in the meantime, all those pointers will be invalid, and
> > we have bugs.  So it looks like, if we want to allow concurrent access
> > to buffers from several threads, we will have a lot of code rewriting
> > on our hands, and the rewritten code will be less efficient, because
> > it will have to always access buffer text via buffer positions and
> > macros like FETCH_BYTE and fetch_char_advance; access through char *
> > pointers will be lost forever.
> 
> Not necessarily lost. We should provide facilities to prevent buffer
> from being modified ("write mutex").

That again means only one thread can access a given buffer, the rest
will be stuck waiting for the mutex.

> This problem is not limited to buffers - any low-level function that
> modifies C object struct must enforce the condition when other threads
> cannot modify the same object. For example SETCAR will have to mark the
> modified object non-writable first, set its car, and release the lock.
> 
> So, any time we need a guarantee that an object remains unchanged, we
> should acquire object-specific write-preventing mutex.

So we will allow access to many/all objects to only one thread at a
time.  How is this better than the current threads?

> Of course, such write locks should happen for short periods of time to
> be efficient.

How can this be done in practice?  Suppose a Lisp program needs to
access some object, so it locks it.  When will it be able to release
the lock, except after it is basically done?  because accessing an
object is not contiguous: you access it, then do something else, then
access it again, etc. -- and assume that the object will not change
between successive accesses.  If you release the lock after each
individual access, that assumption will be false, and all bets are off
again.

> > So maybe we should take a step back and consider a restriction that
> > only one thread can access a buffer at any given time?  WDYT?
> 
> Buffers are so central in Emacs that I do not want to give up before we
> try our best.

But in practice, what you suggest instead does mean we must give up on
that, see above.

> Alternatively, I can try to look into other global states first and
> leave async buffer access to later. If we can get rid of the truly
> global states (which buffer point is not; given that each thread has an
> exclusive lock on its buffer), we can later come back to per-thread
> point and restriction.

That's up to you, although I don't see how the other objects are
different, as explained above.