Re: cc-mode fontification feels random

[Daniel Colascione]

On June 4, 2021 12:11:35 PM Eli Zaretskii <eliz@gnu.org> wrote:

> > Cc: Eli Zaretskii <eliz@gnu.org>, emacs-devel@gnu.org
> > From: Daniel Colascione <dancol@dancol.org>
> > Date: Fri, 4 Jun 2021 11:36:05 -0700
> >
> > On 6/4/21 11:25 AM, Stefan Monnier wrote:
> > > > But I don't understand what's stopping these tree-sitter C modules (like
> > > > [1] and [2]) to have access to the buffer's contents directly and have
> > > > the best of both worlds.
> > > I think it's a direct result of them being "modules": the API doesn't
> > > let modules access a buffer's content directly, so it's more efficient
> > > copy the content via `buffer-substring` and toss it on the other side
> > > than having to use something like `char-after`.
> >
> > The problem is more fundamental than that. Internally, each buffer has a
> > gap. External tools that operate on char arrays don't expect a gap.
> > (They also don't expect to operate on Emacs internal coding, but that's
> > another issue.) If we *did* grant direct buffer access via modules, we'd
> > at least have to memcpy half (on average) the buffer to close the gap,
> > then memcpy half the buffer (on average) to open the gap again when we
> > began editing.
>
> I see no reason for copying, nor for making these tools aware of the
> gap.  At least tree-sitter allows the application to provide a
> function through which tree-sitter will access the edited text.  It
> should be simple to write such a function, because on the C level we
> always know where the gap is.

So you propose providing a "char get_buffer_char(size_t POS)" function? 
That *is* copying If you run that over all values of POS, all you've done 
is make a slow and shitty memcpy.

So you want to amortize the call over several characters? Okay. Now you've 
reinvented buffer-substring.

> > Besides, memory copies are really, really, ridiculously fast. My system
> > can cat from /dev/zero to /dev/null at ~18GB/sec. Copying a buffer's
> > contents so we can give it to tree-sitter should be no issue at all.
>
> Why copy at all? all these libraries need is access to buffer text.
> We can just give it to them.

Because any kind of "access" to the buffer that doesn't expose the gap is 
going to be a copy anyway.