bug#23501: Non-regex-based syntax highlighting

[Eli Zaretskii]

> From: Nir Friedman <quicknir@gmail.com>
> Date: Tue, 10 May 2016 16:16:03 -0400
> Cc: 23501@debbugs.gnu.org
> 
> My idea for a hook was basically to make it possible to provide a callback 
> function to the Major mode. If this
> callback function is provided, then when a new file is loaded or an existing 
> one saved with modifications, the
> callback function is called with the full path to the file.

The syntax highlighting should change also when you modify the buffer,
not only when you save it.  How will that work with your proposed hook?

> The callback function must return something that
> basically tells the major mode how to color everything. A simple way would 
> just be to return a list of the colors
> for every single non-whitespace character taken sequentially. A single very 
> fast pass through this list would
> then be able to color every character.

The hook cannot return a color, because the colors are defined via
faces.  It should return faces instead.

> Is there a reason why that would not be workable?

Maybe it is workable, but you are missing too many details of how
syntax highlight works in Emacs.  As I wrote previously, I encourage
you to study how that works, in order for the proposal to be workable
and practical.

> Also, can you point me to where exactly (e.g. via link to the
> emacs github mirror) the major modes are stored?

It's not the major mode that you need to look at, it's the font-lock
machinery.  Major modes just use the font-lock features by setting the
font-lock faces on portions of the buffer.  Then at display time, the
visible portion of the buffer are displayed as specified by those
faces.  You will see that each major mode simply sets the font-lock
faces, and leaves the rest to the core features.

See font-lock.el and font-core.el for the font-lock features, and
jit-lock.el for the JIT coloring of the visible portions of the
buffer.