bug#56682: Fix the long lines font locking related slowdowns

[Ihor Radchenko]

Gregory Heytings <gregory@heytings.org> writes:

>> Cache existence is not guaranteed.
>>
>
> Can that cache not be created unconditionally when the Org file is opened?

Without cache, the parser will have to process everything on every
request to determine object at point.

>> When cache does not exist, Org must widen. If not, the cache will be 
>> valid only for the current narrowing.
>>
>> When cache do exist, Org may not need to widen. But that also mean that 
>> widening has happened in the past when the cache was generated.
>>
>> At the end, Org does need to be able to widen safely. At least in some 
>> cases.
>>
>
> That will always be possible.  What should better be avoided is to use 
> widen (and to scan the whole buffer) in performance-critical places, such 
> as fontification-functions.

We tried to. In fact, public Org repository does use regexp search and
tries to scan as little as possible inside fontification-function.

However, that approach proved incorrect in a number of occasions. We
have a stable stream of bug reports related to incorrect fontification.
Some were fixed by creating more intricate heuristics, but we reached
the point when using the parser can be actually faster compared to the
fragile heuristic tower in our fontification code.

Our parser, in fact, does not really scan the whole buffer from the very
beginning to determine object at point. It only parses the minimal
possible portion that is sufficient to determine syntax object at the
requested position. This scan is very fast and becomes even faster with
caching. But it does need to look at the whole buffer because Org syntax
is context-dependent. Previous objects outside narrowing can alter the
syntax inside.

-- 
Ihor Radchenko,
Org mode contributor,
Learn more about Org mode at https://orgmode.org/.
Support Org development at https://liberapay.com/org-mode,
or support my work at https://liberapay.com/yantar92