bug#16875: python, comint-mode: Large output makes Emacs freeze

[Stefan Monnier]

>> http://debbugs.gnu.org/cgi/bugreport.cgi?bug=13675
> To a certain extend.
> There some more operations done by comint, which aren't needed when
> executing Python code from source but may slow down it.

I expect the main slowdown comes from font-lock.  But indeed, it''d be
good to try and profile it to see where the time is spent.

Last time I tried to speed up M-x grep, I changed compile.el so that it
only processes (font-lock and friends) the output up to the last \n.
This way, when a very long line is received in 100 chunks, it doesn't
get re-processed 100 times.

Maybe comint.el could do the same (especially since it already assumes
that "text between the last \n and EOB is a prompt").

> Thus solution in python-mode.el is to surpass comint-mode completely and use
> "start-process" and "process-send-string"

If the problem is really in "unneeded comint functionality" (or call it
"bloat"), that's an option, but of course if we don't know where the
performance problem comes, we may end up with the same problem anyway.

> Indeed, "pprint" should solve it already. Maybe make that the default?
> IMO looks better anyway.

[ I don't use Python, so I don't have an opinion on that, but if it
  works, fine by me.  ]

Maybe comint.el could also be changed so that it "wraps" lines if they
get past some arbitrary maximum length (like 10K chars, for example).
It wouldn't be a great solution, but if the performance sucks really bad
past 10K chars, wrapping the line might be a lesser evil.


        Stefan