Re: Emacs Lisp's future

[Søren Pilgård]

On Sun, Oct 9, 2016 at 2:03 PM, Toon Claes <address@hidden> wrote:
> On 7 Oct 2016, at 19:07, John Wiegley <address@hidden> wrote:
>
>  It would be nice to have real compilation
> of Emacs Lisp after the manner of Common Lisp, and maybe the concurrency
> support;
>
>
> Oh yes, please add concurrency support!
> Would it help if we would start a crowdfunding campaign for this? Because I
> know a few fellow emacs user who would definitely back it.
>
> Regards,
> Toon Claes

The thing about concurrency is that it is very hard to integrate
meaningfully and robustly into Emacs.
All Emacs lisp code has the basic assumption, that when it is running,
it is the only code running.
Introducing concurrency would either break that assumption or have to
make it explicit somehow, potentially breaking tons of existing code.

As an example, lets say you have a plugin that count occurrences of a
word, and another that corrects spelling mistakes.
As it is now, the counting plugin can assume no other code runs when
it counts and it can thus simply scan the buffer.
But if we let both run simultaneously the spelling correcter could
modify the buffer resulting in the wrong count as some were done
before and some after the modification.
Oh, but couldn't we then just let each buffer have its own "world"
where everything is sequential?
Well we would lose the ability for the spelling correcter to run while
we were working negating the benefit.
And further, what if the plugin counting occurrences gave the total
count for all buffers?
Then it would have to lock all buffers somehow, so no other code could
modify them simultaneously.

As you can see, introducing concurrency has wide implications for Emacs.
Finding a way to go about it that will still allow old "naive" code to
function correctly is what makes it so tough.
Hopefully we will someday have that, but it requires a lot of hard work.
Maybe we could start out with something like javascripts webworkers. A
function running in its own world without access to buffers or other
shared state, and then we could communicate by message passing.
But as the keen eye would observe that is very similar to simply
spawning an external process.
And it wouldn't benefit much for problems that rely on buffers or global state.

This problem will clearly not be solved just by switching the
underlying platform.