Re: How to add pseudo vector types

[Clément Pit-Claudel]

Thanks for the detailed reply.

On 7/17/21 3:16 AM, Eli Zaretskii wrote:
> When Emacs moves or enlarges/shrinks the gap, that affects the entire
> buffer text after the gap, regardless of where the gap is.  So it will
> affect the TS reader if it reads stuff after the gap.

Doesn't enlarging the gap require allocating a new buffer and copying data to 
it?  If so it wouldn't affect the TS reader.  Moving is indeed trickier, that's 
what I referred to as "limited contention".

>> You do need to be careful to not read the garbage data from the gap, but 
>> otherwise seeing stale or even inconsistent data from the parser thread 
>> shouldn't be an issue, since tree-sitter is supposed to be robust to bad 
>> parses.
> 
> What would be the purpose of calling the parser if we know in advance
> it will fail when it gets to the "garbage" caused by async access to
> the buffer text?

It won't fail, will it?  I thought this was the point of TS, that it would 
reuse the initial parse on the "good" parts in subsequent parses.

> So I don't see how this could be done without some inter-locking.

Yes, there probably need to be some care around the gap area.  But that's what 
I was referring to re. "optimistic concurrency".

> And what do you want the code which requested parsing do while the
> parse thread runs?  The requesting code is in the main thread, so if
> it just waits, you don't gain anything.

You'd have the parser running continuously in the background, every time there 
is a change.  When a piece of code requests a parse it blocks and waits, but 
presumably for not too long because a very recent previous parse means that the 
blocking parse is fast.

>> In fact, depending on how robust tree-sitter is, you might even be able to 
>> do the concurrency-control optimistically (parse everything up to close to 
>> the gap, check that the gap hasn't moved into the region that you read, and 
>> then resume reading or rollback).
> 
> I don't understand what you suggest here.  For starters, the gap could
> move (assuming you are still talking about a separate thread that does
> the parsing), and what do we do then?

Nothing, we start the next parse when this one completes.

> I don't understand what could recording the gap solve.  The stuff in
> the gap is generally garbage, and can easily include invalid multibyte
> sequences.  I don't think it's a good idea to pass that to TS.  Also,
> recording the gap changes in the main thread and accessing that
> information from a concurrent thread again opens a window for races,
> and requires synchronization.

This list of gap changes wouldn't be accessed concurrently: you would 
(message-)pass a copy of it to the parser thread every time it starts a new 
parse.

> Bottom line, I think what you are suggesting is premature
> optimization: we don't yet know that we will need this. 

I thought we knew that a full parse of some files could take over a second; but 
yes, it will be nice if we can find a synchronous way to avoid having to do a 
full parse.