Re: Tokenizing

[Stefan Monnier]

>>> - it should be possible to completely replace the font-lock-mode with
>>> this mode, given a concrete language tokenizer;
>> The intention for font-lock highlighting is to show
>> syntactic structure.  In many cases, we end up limited to lexical
>> highlighting, but that's a problem.
> syntax-propertize provides hooks to let modes add to that;

Maybe you use it for that, but it's definitely not its intention:
`syntax-propertize' is not meant for highlighting at all.


        Stefan