Re: Tramp with global-auto-revert-mode.

[David Kastrup]

Kai Grossjohann <address@hidden> writes:

> David Kastrup <address@hidden> writes:
> 
> > Uh, much too complicated.
> 
> I'm afraid I don't understand you: Do you mean that just the last
> step is too complicated, or that the whole procedure that I described
> is too complicated?

The whole procedure.

> At some point, I thought I understood what you were suggesting, but
> now I'm not sure anymore.
> 
> Let me try to describe my understanding with less detail, then please
> tell me if I got it at least at this level.
> 
>     Whenever Tramp is invoked, it looks to see if the connection
>     buffer is busy.  If it is, it knows that Tramp is interrupting
>     itself, so to speak.
> 
>     If this is the case (Tramp is interrupting itself), Tramp "takes
>     over" the command in progress, fetches all output from it and
>     stashes it someplace safe.

Look, you are talking all the time about "Tramp" as if it was a
sentient being.  It isn't.

"Tramp" consists of two entirely different pieces: the user level
routines (of which several can be running at once in different
"threads" or Emacs' equivalence to it) and the filter routine.  Those
are basically independent from each other.

In order avoid unnecessary lockup in process-send-string, and
problems with identifying responses and input, we might not just
indiscriminately call process-send-string when there are already
outstanding commands.

Ok, here is what the filter routine does when it is called: it
collects the stuff from the output until it has a completely reply to
the currently sent command available.  If it has, it takes the
request and marks it as completed (tacking the results to the
request).

[Entry point for getting a command on the way:]
It then takes a look whether there are still outstanding commands in
the queue.  If there are, it takes the next one from the queue and
sends it through, marking it as being in progress.

That's all.  The filter routine never changes, it does just that.
There is only one filter routine at work at most at any time.

Now for the user level stuff: it knows it needs to get commands
through.  So it makes a request data structure and tacks it to the end
of the current queue (or, if the command is particularly urgent, like
when we are doing autorevert checking, to the _front_ of the current
queue) and then calls accept-process-output on the process repeatedly
until the command finally is marked as being processed.  Then it
takes the results and returns.

That is all.

>     Then it does its own thing.
> 
>     Afterwards, it gets the output from the safe place, inserts it
>     into the connection buffer, and makes it appear to the
>     "interrupted" "master" Tramp as if nothing had happened in the
>     meantime.
> 
> You were also talking about a queue of commands, which I'm not
> seeing.  So this would be an indication that I misunderstood you.

Probably.  You don't just want to stuff the pipe with
process-send-string indiscriminately, it may stall at one time, and
then a different thread might feel tempted to stuff its material
right in the wrong place.  So at any time, at most one thread may be
allowed to send strings.  If there is already a reason for the filter
routine to run (because some command is in the process of being
transmitted), then the requesting process will _not_ write to the
tty, but meekly queue his request into a request queue and hope that
the filter routine will at some time be friendly enough to process it.

> (The above does imply a stack of commands, corresponding to
> interruptions of interruptions.)

Quite.  Since Emacs does not have true concurrency, "deeper" commands
in the "user stack" will not get resumed or finished until "higher"
commands are completed.  Which they eventually will, since the filter
routine will cater for everything in the queue.  For stuff like
autorevert it would be a good idea to check whether the queue already
contains some request for the same thing.  If it does, no need to add
another one.  Similar things hold for fetching a particular directory
into the directory buffer, although here the resulting information
_is_ needed, so one needs to call accept-process-output until the
information has arrived, even though it was requested by a different
thread/layer/task/whatyouwantocallit.

> > When output arrives, the currently active filter function will be
> > called with it and _all_ accept-process-output calling threads
> > will get woken up again.  So they will need to check whether their
> > respective work has already been done or aborted by the filter
> > routine.
> 
> Hm.  If more than one thread of execution will be woken up at the
> same time, then surely the above procedure won't work.

Emacs can't do anything "at the same time".  We don't have
concurrency.  It has a stack of outstanding commands started from the
main loop (and a list of things it may call from the main loop).  It
works this stack off top to bottom.

Even if Emacs had separate stacks for the main loops, it would not
schedule between them except when control is explicitly yielded (can
happen with I/O).

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum