Re: cc-mode fontification feels random

[Ergus]

On Thu, Jun 10, 2021 at 05:34:31PM +0200, �scar Fuentes wrote:
> Ergus <spacibba@aol.com> writes:
>
> > For things like font-lock/display engine I only consider to do as much
> > as possible in the C side to improve performance. And reduce as much as
> > possible interacting with the lisp side... Do you think that it may be
> > possible?
>
> Before going this route, we need to check if native-comp is enough of an
> improvement and, if it isn't, try to improve it.
I work very extensively with jit compilers and similar and with
different architectures (ARM, ePIC, Intel).

In my experience the JIT improvement in performance is very significant
compared to bytecode. Specially for a similar code the difference in
time can be 1 or even 2 orders of magnitude better.

BUT

When translating from high level languages (my experience: cpython and
julia) it requires much more effort, optimization and time to improve
the compiler to get just a same order or performance than a similar C
code. Just creating a low level "intrinsic" or "binding" saves time and
relies in many other optimization the C compiler already have.

Ex: AOS vs SOA, vectorization, parallelization and similar optimizations
are very easy to do at low level (or give the hints to the compiler to
do them). But it is extremely hard to teach a high level compiler to do
themq; basically because of the data structures and types we use in high
level languages.

That's why in python it is so extended to use libraries like Pandas or
Numpy. And every time more and more python packages are just interfaces
to C libraries. Julia on the other hand provides C primitives for
everything and has primitive data types to give more hints to the
compiler... but even with that in real code it can't compare to
Python+numpy. Other languages like PHP have a very good compiler
improved for decades, but even with that, they have moved a lot of their
functionalities to C code with some bindings.

Font locking is a dynamic feature and affects responsiveness, and must
be executed in the background constantly, so even fast will be never too
fast. Responsiveness is an usual complain when new users come from
different editors. But also the languages syntax are expected to become
more complex with the time.

In our case, just accessing the buffer content and passing directly to
tree-sitter in C will be almost trivial at the low level, without types
conversions or extra copies; but also when we receive the output,
processing them with the json library we already link against will be
orders of magnitude simpler and faster, instead of converting them to
lisp object and stressing the gc with temporal objects we don't really
need, and then iterate on the lisp level.

Sadly not all the architectures are supported by libgccjit either. So a
low level call-preprocess solution will work faster for
everyone. Specially for those that still use emacs 23 due to the speed
feeling.