bug#45938: 28.0.50; python native completion fails with fancier readline settings

[Zoltán Vandrus]

Three problematic options I found are:

   set colored-stats on
   set colored-completion-prefix on
   set enable-bracketed-paste on

With any of these M-x run-python warns:

   Warning (python): Your ‘python-shell-interpreter’ doesn’t seem to
   support readline, yet ‘python-shell-completion-native-enable’ was t and
   "python3" is not part of the
   ‘python-shell-completion-native-disabled-interpreters’ list.  Native
   completions have been disabled locally.
   

set colored-stats on
set colored-completion-prefix on
================================

Any of these on and *Python completions redirect* buffer ends with:
0__dummy_completion__^[[0m^[[K  1__dummy_completion__^[[0m^[[K

^[[0m and ^[[K are escape sequences not understood by comint mode I
think

Related bug: #24401 recommends setting colored-stats conditionally only
for bash, because filename coloring is not that useful for the python
shell.

Even then colored-completion-prefix is useful everywhere, emacs does it
by default too.

set enable-bracketed-paste on
=============================

Fails like half of the times for me. For a different reason.

First record output:
   (defun keep-output (output)
     (setq kept (cons output kept)))

   (add-hook 'inferior-python-mode-hook
             (lambda ()
               (setq kept nil)
               (add-hook 'comint-output-filter-functions #'keep-output)))

M-x run-python

kept’s value is
("python.el: native completion setup loaded
^[[?2004h>>> " "^[[?2004l^M" "Python 3.9.0 (default, Oct  7 2020, 23:09:01) 
[GCC 10.2.0] on linux
Type \"help\", \"copyright\", \"credits\" or \"license\" for more information.
^[[?2004h>>> ")

-----------------------

python-shell-completin-native-setup assumes that after running the
python script that sets up completion the first output will be from the script:
      (when (and
             (python-shell-accept-process-output
              process python-shell-completion-native-try-output-timeout)
             (save-excursion
               (re-search-backward
                (regexp-quote "python.el: native completion setup loaded") nil 
t 1)))
        (python-shell-completion-native-try)))))

What can happen sometimes is that, after the first initial prompt:
   "Python 3.9.0 (default, Oct  7 2020, 23:09:01) 
   [GCC 10.2.0] on linux
   Type \"help\", \"copyright\", \"credits\" or \"license\" for more 
information.
   ^[[?2004h>>> "
The python script is injected.
python-shell-accept-process-output waits for next prompt
Bracketed mode is disabled instead with clearing the line: "^[[?2004l^M"
Empty line is a valid python prompt, so it's accepted, and the regexp
test fails.

#41959 is another problem with trying to parse the readline output.

--------------------------------------------------

Readline version 8.1.0

In GNU Emacs 28.0.50 (build 4, x86_64-pc-linux-gnu, GTK+ Version 3.24.24, cairo 
version 1.17.4)
 of 2021-01-17 built on zoli-laptop
Repository revision: 1773679af3241919a85d6174b1554070a63cca79
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12010000
System Description: Arch Linux

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSYSTEMD LIBXML2 M17N_FLT MODULES NOTIFY INOTIFY
PDUMPER PNG RSVG SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM
GTK3 ZLIB

Important settings:
  value of $LC_TIME: C
  value of $LANG: hu_HU.utf8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  shell-dirtrack-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc puny dired dired-loaddefs
rfc822 mml mml-sec epa derived epg epg-config gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils warnings compile text-property-search
cl-extra help-mode python easymenu tramp-sh tramp tramp-loaddefs
trampver tramp-integration files-x tramp-compat shell pcomplete
parse-time iso8601 time-date ls-lisp format-spec auth-source cl-seq
eieio eieio-core cl-macs eieio-loaddefs password-cache json subr-x map
seq byte-opt gv bytecomp byte-compile cconv comint ring cl-loaddefs
cl-lib ansi-color iso-transl tooltip eldoc electric uniquify ediff-hook
vc-hooks lisp-float-type mwheel term/x-win x-win term/common-win x-dnd
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode elisp-mode lisp-mode prog-mode register page
tab-bar menu-bar rfn-eshadow isearch timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core term/tty-colors frame
minibuffer cl-generic cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite charscript charprop case-table epa-hook jka-cmpr-hook help
simple abbrev obarray cl-preloaded nadvice button loaddefs faces
cus-face macroexp files window text-properties overlay sha1 md5 base64
format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify lcms2 dynamic-setting
system-font-setting font-render-setting cairo move-toolbar gtk x-toolkit
x multi-tty make-network-process emacs)

Memory information:
((conses 16 74431 8365)
 (symbols 48 9112 1)
 (strings 32 27268 1190)
 (string-bytes 1 989045)
 (vectors 16 15840)
 (vector-slots 8 208279 9279)
 (floats 8 47 29)
 (intervals 56 226 0)
 (buffers 984 14))