bug#47657: python-shell font-lock with multi-line input: runaway fontification buffer length

[JD Smith]

`python-shell-font-lock-post-command-hook’ makes the presumption that only a single line of input can appear at the prompt.  It includes this preparatory “clean-up” of its font lock buffer after first having copied all input lines after the prompt:

      (let* ((input (buffer-substring-no-properties

                     prompt-end (point-max)))

           ...

             (replacement

              (python-shell-font-lock-with-font-lock-buffer

                (delete-region (line-beginning-position)

                               (point-max))

                (setq font-lock-buffer-pos (point))

                (insert input)

This means that using, e.g., C-c SPC (`comint-accumulate’) to insert a newline at the shell prompt leads to runaway growth of the font-lock buffer (“ *Python-font-lock”, by default).  Any command, like an insert, produces a new copy of all but the final line of input in this buffer.  Since this entire buffer is font-locked from scratch after every single command, this becomes very inefficient.  

Find an example, below.

The simple fix would be to delete the entire font-lock buffer file contents:

(delete-region (point-min) (point-max))

A perhaps smarter and more efficient fix would be to enable font locking in the inferior Python buffer itself, but constrain it to the text after the process mark.  This would also prevent front-lock from constantly having to re-fontify the entire input text after each command. 

++++

(In *Python* shell buffer):

In [2]: a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

d='and I’ 

(In " *Python-font-lock"; this can become arbitrarily long):

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

a= 'now is the time'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

a= 'now is the time'

for i in range(12):

a = 12; print(i+4-4) 

b='one two one two and through and through'

d='and I'