Re: Using incremental parsing in Emacs

[Eli Zaretskii]

> Date: Sat, 4 Jan 2020 12:57:24 +0800
> From: HaiJun Zhang <address@hidden>
> 
> For font-lock, I think it may work with tree-sitter like this:
> 1. After openning a file, parse the whole buffer with tree-sitter. We get a 
> syntax tree from tree-sitter.
> 2. Get the syntax nodes with ts_node_descendant_for_point_range and fontify 
> the buffer text in the visible
> region or whole buffer.
> 3. After each modification of buffer text, make a copy of the syntax tree as 
> the new one. Update the new one
> with the modification.
> 4. Get the changed range and changed nodes list by comparing the old and new 
> syntax trees. Then free the
> old syntax tree.
> 5. Update the text properties in the changed range.
> 6. Goto 3

I encourage you to study how JIT font lock works in Emacs, and in
particular how it plugs itself into the display engine.  Because using
any incremental parsing technology for font-lock needs a good
understanding of how font-lock is typically used in Emacs, and any
practical suggestions for integration and interfaces must take that
into consideration.

E.g., step 1 is anathema to JIT font-lock: it would produce a long
delay in displaying a file's buffer when the file is first visited.
For example, think about visiting a large and complex source file such
as xdisp.c: even if it takes tens of milliseconds to parse all of it,
as some tree-sitter presentation claims, waiting for that long before
we even start displaying the first window-full would be an annoyance.
And that's even before we consider the time to compute all the face
text properties from the syntax tree, something that will also take
time.

Thanks.