Re: Using incremental parsing in Emacs

[arthur miller]

Eli Zaretskii <address@hidden> writes:

>> Date: Sat, 4 Jan 2020 12:57:24 +0800
>> From: HaiJun Zhang <address@hidden>
>> 
>> For font-lock, I think it may work with tree-sitter like this:
>> 1. After openning a file, parse the whole buffer with tree-sitter. We get a 
>> syntax tree from tree-sitter.
>> 2. Get the syntax nodes with ts_node_descendant_for_point_range and fontify 
>> the buffer text in the visible
>> region or whole buffer.
>> 3. After each modification of buffer text, make a copy of the syntax tree as 
>> the new one. Update the new one
>> with the modification.
>> 4. Get the changed range and changed nodes list by comparing the old and new 
>> syntax trees. Then free the
>> old syntax tree.
>> 5. Update the text properties in the changed range.
>> 6. Goto 3
>
> I encourage you to study how JIT font lock works in Emacs, and in
> particular how it plugs itself into the display engine.  Because using
> any incremental parsing technology for font-lock needs a good
> understanding of how font-lock is typically used in Emacs, and any
> practical suggestions for integration and interfaces must take that
> into consideration.
>
> E.g., step 1 is anathema to JIT font-lock: it would produce a long
> delay in displaying a file's buffer when the file is first visited.
> For example, think about visiting a large and complex source file such
> as xdisp.c: even if it takes tens of milliseconds to parse all of it,
> as some tree-sitter presentation claims, waiting for that long before
> we even start displaying the first window-full would be an annoyance.
> And that's even before we consider the time to compute all the face
> text properties from the syntax tree, something that will also take
> time.
>
> Thanks.

Do it in a thread and display file originally without syntax coloring,
and then gradually display results as the tree-sitter thread work it's
way?

Maybe start with displayed portion of the file only. Then since
tree-sitter accepts changes as fine-grained as on character level, send
new lines for syntax colouring as they are scrolled upp or down.

Or other strategy could be to have tree-sitter thread highlight the
visible portion of the file and to continue to work with non-visible
part of the buffer in background?

Could something like that work?