bug#57207: 29.0.50; Fontification is slow after e7b5912b23 (Improvements to long lines handling)

[Eli Zaretskii]

> Cc: 57207@debbugs.gnu.org
> Date: Sun, 14 Aug 2022 16:08:27 +0000
> From: Gregory Heytings <gregory@heytings.org>
> 
> > I am experiencing severe performance degradation on current master.
> >
> > The degradation happens when working with Org files
> >
> > [...]
> >
> > I was able to trigger the slowdown when unfolding headlines hidden using 
> > 'invisible text property.
> >
> > Do you have any idea what could be going wrong or how to debug things 
> > further?
> 
> Can you (a) provide a MRE so that we could track down what's going on 
> and/or (b) try to set long-line-threshold to a larger value (20000, 50000, 
> 100000) and see when the performance degradation disappears?

Gregory, I suspect that the problem could be in the loop in
redisplay_window, where we look for long lines.  Changing visibility
in Org changes text properties, and that causes MODIFF to be
incremented.  I actually can cause something similar without Org at
all, just by scrolling fast through xdisp.c.  You can put a breakpoint
inside the 'if' that guards the re-evaluation of the long-lines in
redisplay_window, and scroll with C-v through xdisp.c immediately
after visiting it -- I hit the breakpoint from time to time, although
there are no changes to the buffer except faces placed by
fontifications.

jit-lock runs under with-silent-modifications, but that only adjusts
SAVE_MODIFF to create an illusion of unchanged buffer, it doesn't
affect UNCHANGED_MODIFIED.

Maybe we should measure the increment in MODIFF across the calls to
fontification-functions in handle_fontified_prop, and correct
UNCHANGED_MODIFIED by the same increment when fontification-functions
return?

Ihor, if you set long-line-threshold to a nil value, do the problems
go away?  If they do, can you put a breakpoint in xdisp.c here:

  /* Check whether the buffer to be displayed contains long lines.  */
  if (!NILP (Vlong_line_threshold)
      && !current_buffer->long_line_optimizations_p
      && MODIFF - UNCHANGED_MODIFIED > 8)
    {
      ptrdiff_t cur, next, found, max = 0, threshold;
      threshold = XFIXNUM (Vlong_line_threshold);  <<<<<<<<<<<<<<<<<<<<<<<<<<<
      for (cur = BEG; cur < Z; cur = next)
        {
          next = find_newline1 (cur, CHAR_TO_BYTE (cur), 0, -1, 1,
                                &found, NULL, true);
          if (next - cur > max) max = next - cur;
          if (!found || max > threshold) break;
        }
      if (max > threshold)
        current_buffer->long_line_optimizations_p = true;
    }

and see how frequently this gets called in the same buffer when you do
whatever shows the performance degradation?