bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii]

> Date: Fri, 05 Aug 2022 09:37:35 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: gerd.moellmann@gmail.com, 56682@debbugs.gnu.org, 
> monnier@iro.umontreal.ca, 
>     dgutov@yandex.ru
> 
> > It doesn't surprise me.  If you disable font-lock and show-paren-mode, 
> > does it become significantly faster?  And how does disabling font-lock 
> > that measure vs disabling auto-composition-mode with that file?
> 
> Disabling show-paren-mode and font-lock has no effect, no.  What is 
> surprising is that the speed seems to depend both on the mode and on the 
> presence of bidirectionality (so it seems that after all bidi is involved 
> in one way or another).

That the presence of R2L characters makes this slower shouldn't
surprise.  The bidi iteration through buffer text _can_ be non-linear,
but it's only _actually_ non-linear when R2L characters are present.
So the complications with handle_stop_backwards and
compute_stop_backwards actually happen only when there are R2L
characters in the buffer; otherwise the iterator behaves exactly like
the unidirectional display in Emacs 23 and before did: it examines
characters one by one it strict increasing order of buffer positions,
and skips or shortcuts some processing that is only needed for truly
bidirectional text.

IOW, the display engine is specially optimized for the very frequent
case of buffers that don't include R2L characters, and is expected to
be slower otherwise.

If font-lock doesn't produce any tangible difference, it probably
means that the amount of stop-positions added due to faces is
negligible when compared to the stop-positions due to character
composition.  Which I guess is also reasonable with a script like
Arabic.

> 1. https://www.heytings.org/data/arabic-large.json
> 2. https://www.heytings.org/data/arabic-large.json.txt
> 3. https://www.heytings.org/data/arabic-large.txt
> 4. https://www.heytings.org/data/arabic-large.txt.json
> 5. https://www.heytings.org/data/arabic-small.json
> 6. https://www.heytings.org/data/arabic-small.json.txt
> 7. https://www.heytings.org/data/arabic-small.txt
> 8. https://www.heytings.org/data/arabic-small.txt.json
> 
> 1 and 2, 3 and 4, 5 and 6, 7 and 8 are the same file, the only difference 
> is the added extension.  1, 2, 3 and 4 on the one hand, and 5, 6, 7 and 8 
> on the other hand, are almost the same file, the only difference is that 
> in 1 and 2 and 5 and 6 the arabic text is enclosed into '{"ar":"<arabic 
> text>"}'.
> 
> Now what you'll see is that 5 is slow, 6 is also slow (so it's not only 
> js-mode which is doing something wrong), 7 is fast, and 8 is again slow 
> (so js-mode is perhaps doing something wrong).  Also, the motion commands 
> C-n and C-p do not work as expected in 5, 6 and 8.

Thanks, I will use these.

There's (at least) one more aspect of this, as long as Text mode is
being used: Text mode doesn't force bidi-paragraph-direction to be
left-to-right, whereas all descendants of prog-mode, including
js-mode, do.  Leaving bidi-paragraph-direction at nil means Emacs
needs to determine the base paragraph direction each time it's about
to redisplay a window, and that might be expensive, especially in a
large buffer without any paragraph breaks (by default, an empty line),
because that is determined by the first strong directional character
of the paragraph.  So for a more fair comparison with Text mode, you
should set bidi-paragraph-direction to the value left-to-right in
text-mode buffers.