bug#61350: Eglot over Tramp freezes with large project

[João Távora]

Michael Albinus <michael.albinus@gmx.de> writes:

>> Michael can probably confirm, correct or deny this.
> More or less correct.

Actually, I don't think it is correct, at least not the way I meant it:
JSONRPC data never gets into the tprocess's buffer.  Experiments also
seem to disprove it.

> But I still can't say which process gets output
> when, because I cannot debug accept-process-output (it's a C
> function). And running Emacs under gdb changes timings, which is
> important I believe.

Yes, we must probably write some gdb scripts.  Eli is expert at that,
but I've done some it myself.

>> When one (accept-process-output tprocess nil nil 'JUST-THIS-ONE=t) one
>> must be absolutely sure that tprocess is going to send _something_
>> "soon".  If it doesn't, we'll hang indefinitely (until the process dies
>> or the user quits)
>
> Yes. But Tramp calls accept-process-output only, if it has send a
> command to the remote shell, and it expects something to be returned. At
> least the shell prompt.

Yes.  Tramp is doing the right thing.  It really expects a response to
come.  And more often than not, it does.  But sometimes it doesn't, and
that's when we hang.

>> See the comment there?  Only 256 characters back are inspected.
>
> Yes. But the regexp it searches for is the final shell prompt. Something
> like "///4b3b7d4fa561141e84c94a1cf25e8575#$", which is shorter than 256
> bytes for sure.

OK, but why can't one search for it from where you last left parsing,
(i.e. point) up to process-mark?  Anyway, I increased that value
significantly and it didn't make a difference, so this is probably a red
herring.

>> So, finally, here's my conjecture:
>>
>> 1. Tramp goes into 'tramp-wait-for-regexp'.  tprocess's buffer already
>>    the message that 'found' is supposed to return, but it also has a lot
>>    more stuff, say a lot of JSONRPC data from the LSP server that also
>>    came into that tprocess buffer and is awaiting to be delivered to
>>    jprocess.
>>
>> 2. This data is for piping into jprocess, where the JSONRPC message will
>>    be decoded, but it will probably never arrive at its destination.
>>
>> 3. 'found' will be nil in tramp-wait-for-regexp, because of the
>>    tramp-search-regexp limitation.
>>
>> 4. tramp-wait-for-regexp will issue the "risky" accept-process-output
>>    call.
>>
>> 5. there is no more data that accept-process-output wants to put in the
>>    buffer,  because the LSP server is fine for the moment.
>>
>> 6. Emacs hang
>>
>> Just a conjecture.
>
> Yes, this is more or less the scenario. But I still don't understand why
> not all data are delivered through the socket ssh is using. Could it be
> there is a limitation, how much data could be buffered by ssh?

After testing with a enhanced tramp-backtrace that prints out the
contents of every process buffer, I don't think my conjecture is
correct.

diff --git a/lisp/net/tramp.el b/lisp/net/tramp.el
index df4b7dfca2c..f24a3b51074 100644
--- a/lisp/net/tramp.el
+++ b/lisp/net/tramp.el
@@ -2212,12 +2212,21 @@ tramp-backtrace
 If VEC-OR-PROC is nil, the buffer *debug tramp* is used.  FORCE
 forces the backtrace even if `tramp-verbose' is less than 10.
 This function is meant for debugging purposes."
-  (let ((tramp-verbose (if force 10 tramp-verbose)))
+  (let ((tramp-verbose (if force 10 tramp-verbose))
+        (bt (lambda ()
+              (backtrace)
+              (dolist (p (process-list))
+                (let* ((buf (process-buffer p))
+                       (name (and buf (buffer-name buf))))
+                  (when buf
+                    (princ (format "\n--8<---- begin contents of `%s' 
------>8---\n" name))
+                    (princ (with-current-buffer buf (buffer-string)))
+                    (princ (format "\n--8<---- end contents of `%s' 
------>8---\n" name))))))))
     (when (>= tramp-verbose 10)
       (if vec-or-proc
          (tramp-message
-          vec-or-proc 10 "\n%s" (with-output-to-string (backtrace)))
-       (with-output-to-temp-buffer "*debug tramp*" (backtrace))))))
+          vec-or-proc 10 "\n%s" (with-output-to-string (funcall bt)))
+       (with-output-to-temp-buffer "*debug tramp*" (funcall bt))))))

When you press C-g after the hang occurs, the backtrace is correct but
the tprocess buffer is simply empty, according to the new logs.  IOW the
response Tramp was waiting for never arrived.

>> There are no (usable) threads in Emacs.
> There are. I made Tramp using threads, and it worked fine, when no
> interactive dialogue inside a thread happened.

Right.  There are so-called green threads, which could fit input-waiting
situations like this one, not for achieving true simultaneity.  In my
opinion, they don't present any advantage over the evented model, which
we understand much better (wait... except we don't :-) )

I don't think we should do that until we understand what is happening.

>> Timers are events, and so are runs of each processe's process filter.
>> Those two are what creates asynchronicity and the emulation of
>> simultaneity in Emacs.  When jprocess's filter sees a whole JSONRPC
>> message, it calls the message handler.
>
> Timers and process filters are the cause of the "Forbidden reentrant
> call in Tramp" errors.

I've had problems with reentrant calls to process filters before, and I
solved them with a timer.  Not sure what that error is.

> Wwe must do anything, solving this.

I don't understand what you mean here.

João