bug#19755: python.el: native completion: more problems (and solutions)

[Carlos Pita]

Here is a quick prototype of how the client/server approach would look like.

The setup code could be structured around a single class PythonElHelper, say.

The class will listen on a socket in a dedicated thread.

I have coded its output as json in order to allow to pass structured
information to the elisp side and easily parse this information as
elisp data with json-read-from-string.

I don't see a need to use a full-fledged http server, although that
could be done with url.el if desired.

The server would provide tooltips for eldoc, documentation for the
help buffer and completions for completion-at-point in a way that
doesn't interfere with history, prompt numbers, block editing, etc.
and doesn't require to deal with the shell input/output.

Do you think this approach is sensible? I'm available for coding this
along the next two months or so.

######################## python server #################


class PythonElServer:

    def __init__(self):
        # detect python2/3, ipython, readline, set completer, etc
        pass

    def complete(self, symbol):
        return ["a", "b", "c"]

    def tooltip(self, symbol):
        return "func(a, b, c)"

    def documentation(self, symbol):
        return "A function that does something"

    def serve(self, port):
        import socket, json
        server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        server.bind(('', port))
        server.listen(1)
        (client, address) = server.accept()
        while True:
            request = client.recv(1024)
            action, symbol = request.split()
            response = getattr(self, action)(symbol)
            client.send(json.dumps(response))

PythonElServer().serve(9090)


######################## elisp client #################

;; -*- lexical-binding: t -*-

(setq python-shell-server
      (open-network-stream "pyserver" "pyserver" "localhost" 9091))

(defun python-shell-server-request (action symbol)
  (set-process-filter
   python-shell-server
   (lambda (proc out)
     (let ((response (json-read-from-string out)))
       (pcase action
     (`complete (python-shell-complete response))
     (`tooltip (python-shell-tooltip response))
     (`documentation (python-shell-documentation response))))))
  (process-send-string python-shell-server
               (concat (symbol-name action) " " symbol)))

(defun python-shell-complete (response)
  (print (concat "complete: " response)))

(defun python-shell-tooltip (response)
  (print (concat "tooltip: " response)))

(defun python-shell-documentation (response)
  (print (concat "documentation: " response)))

(python-shell-server-request 'documentation "xxx")