bug#40930: AUCTeX fontification messed up

[gennady.uraltsev]

Dear Everyone,

I have posted a reddit thread about a bug wth AUCTeX fontification that
exists since 6 years or so. I myself reported it here in 2019-06 but it was
ignored. The bug is occurs frequently but is very hard to reproduce
reliably: this is why I can't include a minimal example. The reddit thread
is here:

https://www.reddit.com/r/emacs/comments/g95vyl/font_lock_in_auctex_messed_up
/

and I copy the contents below for completeness. There is some discussion in
the comments there. 
It would be great to have some instructions on how to approach the problem. 

*****
Hello everyone. I use AUCTeX to write papers in LaTeX. It is a great
environment if configured correctly. I use it with preview-latex and
prettify-symbols-mode. I want to use TeX-fold-mode but I have not set it up
yet.

However there is a recurring problem that is very difficult to catch and to
diagnose. I have reported it on the bug-auctex mailing list and google
searches give many other people complaining about the issue:


    https://lists.gnu.org/archive/html/bug-auctex/2019-06/msg00005.html

 
https://emacs.stackexchange.com/questions/20518/how-to-investigate-color-ble
eding

    https://github.com/syl20bnr/spacemacs/issues/9976

    http://pragmaticemacs.com/emacs/refresh-auctex-syntax-highlighting/

    https://github.com/syl20bnr/spacemacs/issues/5433

It seems to be a very difficult issue to reproduce consistently, even though
it appears often in any large enough tex file.

Here is what I know:

It appears in: vanilla emacs, spacemacs, doom emacs.

It appears in: windows, linux (I am currently on windows)

It appears with

(setq TeX-install-font-lock 'tex-font-setup)

and

(setq TeX-install-font-lock 'font-latex-setup)

It *does not* depend on this issue
https://www.gnu.org/software/auctex/manual/auctex/Known-problems.html#Known-
problems

(no verbatim or dollar signs)

The problem can reappear and disappear even by inserting a single letter (in
a math environment or outside one)

The problem can sometimes reappear and disappear by merely scrolling the
buffer

The problem seems to get worse with larger files

The problem seems to get worse with other functionality that relies on
overlays/text properties e.g. preview-latex or rainbow-delimiters but it
presents itself even without these packages.

Another approach is to disable jit. It seems to help a lot but it doesn't
solve the issue. Mistakes still occur. They are just fewer and thus
tolerable.

The problem disappears when doing M-x normal-mode but any action can make it
reappear.

Unfortunately, I am currently writing quite a large paper and this bug is
making AUCTeX unusable. Every several letters I write trigger a change in
fontification.

The developers have ignored my bug report: I don't blame them. It is
completely unclear how to reproduce it. I did ask for instructions on what
to do but none were given.

I am writing here to pool together all I have found about the issue so we
could possibly help track this bug down. Any suggestions?

EDIT:disabling jit helps a bit.

****

Best,

Gennady