Re: Multi-tty design (Re: Reordering etc/NEWS)

[Karoly Lorentey]

David Kastrup wrote:
> Karoly Lorentey <address@hidden> writes:
>> No, my memory has failed me.  I had a patch implementing the above
>> design, but what we currently have in the tree is something more
>> complex: environment variables are neither frame-, nor
>> terminal-local, but rather client-local.
> 
> I have seen on the archives of the multi-tty list and its README that
> the implementation has went through several different ideas.  So quite
> a bit of work has been invested already, and there is obviously not
> going to be much enthusiasm for scrapping it.

Sure, but better scrap parts than drop the whole thing.  As you say,
environments have already gone through several reimplementations.  I
certainly don't mind having another iteration. :-)  What we have now is
my personal favourite, but that doesn't mean it's the best for everyone.

I argue that while it is a valid viewpoint to say that things such as
CFLAGS or TEXINPUTS should always come from the original environment, it
is equally valid and defensible to say that they should come from the
local terminal.  (Emacs may have been running in the background for
weeks, and I may have just started working on my brand new TeX file in a
recently started emacsclient session.)  Both viewpoints should be
catered for.

> I think that "client-local" is a complication we really don't want to
> introduce.

Fair enough.  So let's do terminal-local environments.  I hereby
withdraw the current environment implementation, and propose the
following simple solution instead:

        - Make `process-environment' a terminal-local variable
          (with DEFVAR_KBOARD).

        - Have all environment variables be terminal-local.
          (Keep reading!) :-)

        - Getenv should look at the current binding of
          `process-environment' only.

        - Modify the default behaviour of `setenv' to change the
          variable on all terminals, one by one.  In a loop.

          If it is desired that M-x setenv affect future terminals as
          well, then we can make it keep a list of changed variables and
          apply this list on the terminal-local environment whenever a
          new terminal is initialized.  I suggest doing this.

        - To support having only DISPLAY and a selected few other
          variables differ on separate terminals, we can tweak the
          initialization of terminal-local environment lists to copy the
          rest of the variables from the original environment.  The
          lists will, however, remain separate.

The above solution is easy to implement, easy to explain, doesn't
involve fragile Lisp structures, and does not break any existing code.
That is to say, users using a single terminal will not notice any change
whatsoever.  All code that works in Emacs 22 will work in
single-terminal Emacs 23.  All environment-related regressions are
prevented, period.

Multi-terminal users may notice that some environment variables are
different on different terminals.  This will not lead to confusion,
however, since two frames can only appear side by side when they are on
the same terminal.  (This is an important point.)

Existing Elisp code that does not change to a (literally) randomly
selected frame after temporarily setting up a particular environment
will work without changes when multiple terminals are simultaneously
present in Emacs 23.

One deviation in the multi-terminal case is that code like

        (let ((oldval (getenv "FOO"))
          (setenv "FOO" "fred")
          (unwind-protect
                ...
             (setenv "FOO" oldval))

will change the value of FOO on all terminals but the current one as a
new side effect.  This is, I believe, acceptable.  Some code adaptation
to make existing code able to take advantage of the new feature set is
inevitable.  We can simply document this concern and suggest solutions
(e.g., let-binding `process-environment' instead, or adding a standard
macro that would save and restore `process-environment' on all
terminals.) in the NEWS file, or wherever we provide an upgrade guide
for package writers.

(Here is another example where some (many) existing packages will fail
in a multi-terminal environment: It is very common in Elisp code to have
window-system dependent variable values be initialized at load time.
This works fine in the single-terminal case, but fails (with symptoms of
various severity) in a tty+X multi-terminal environment.)

I feel this is a good compromise between compatibility and new features.
 In the rest of this mail I'm going to argue for the above proposed design.

> Here is how to approach this with a "minimally invasive hack" (namely
> something which works with most existing code, but which one would not
> really want to ever write into a manual): make process-environment a
> terminal-local variable.

So far, so good. :-)

> It will be an nconc of terminal local
> environment variables and `global-process-environment', the
> environment Emacs got started with.

I really don't see a reason to do a thing like that, but I would accept
it if you think it's important, and if you agree to provide an option to
disable it.

[(Very useful) list of reference use-cases]
> So those are the semantics we want to keep if we possibly can.

All of these use-cases will work unchanged in single-terminal sessions.
Some of them will need to be changed to be fully compatible with
multi-terminal sessions.  Very good.

> We also would want to have terminal-local environment variables (like
> "DISPLAY") appear in process-environment.

We can mark this done.

> For some reason it would seem that

(permanent)

> manipulation of process-environment happens almost
> exclusively through setenv.

In the new design, this will work unchanged in multi-terminal sessions
as well.

> So here are a few options:
> 
> a) make process-environment a terminal-local R/W variable.  Notice
> that this does _not_ imply that anything but the first element of the
> list can't be actually shared among the lists.  As long as
> manipulation of process-environment happens with setenv, we are off
> ok.  Changes of existing values can be done with setcar, so that
> terminal-local environment variables (at the start of the list) will
> stay terminal-local and vice versa.

AFAICS, this is my above design, plus tail-merging, minus global setenv.
 I don't hate it.  As I said above, if you'd really prefer, I can live
with the shared tails, provided they can be easily disabled.

> Disadvantage: if somebody manipulates process-environment with setq
> and a copy, the variable set for this terminal will _then_ become
> completely detached from that of other terminals.

This doesn't affect single-terminal users, and will not lead to
catastrophic failure with multiple terminals either.  Most users will
not even notice, and as you see some will even prefer this state as
default. :-)

> b) make process-environment a global variable that can be manipulated
> like the user wants.  However, whenever call-process or similar are
> invoked, the actual environment passed into the called process has a
> set of terminal-local environment settings prepended.
[...]
> It will be a nuisance and might break those programs that manipulate
> the environment variables accessing the tty, but only those.

I think this solution would be both incompatible and much too complex.

[...]
> And it will also cause a lot of
> inconsistencies that can't really be explained well to the user, like
> "compile" behaving differently in windows that are side-by-side.

I have explained above why I think this is an invalid argument with
terminal-local variables.  If two windows are side by side, then they
are on the same terminal.

>>        Terminal-local environments would less complete, but still
>>        good enough to be usable without many problems.
> 
> That's what I would aim for, and only for those variables that are
> indeed terminal-dependent.

OK, so let's do that using my design.  Or yours.

>> If you find client-local environments unacceptably ugly, I can
>> update and submit my patch for simple terminal-local environments.
>> The local environment then becomes a simple terminal parameter,
>> initialized when the terminal is created.  This is a much simpler
>> solution that retains much of the feature set of the current design.
> 
> I think we should aim for the simplest solution that we can reasonably
> explain.  My favorite solution, as explained above,

... is complex, hard to explain and backwards incompatible.

> I don't see how we could avoid this while maintaining
> reasonable upwards compatibility.

There is no 100% upwards compatible solution.  (Except perhaps the ones
using hypothetical future trans-human AI programs to automatically
rewrite existing single-terminal code.  I hope we will manage to release
Emacs 23 before the technological singularity makes human-operated
editors obsolete.  On the other hand, self-awareness, intuition and a
little creativity would be killer features to implement for Emacs 24.
We could then let Emacs 25 write itself, and go on holiday.
But I digress.) :-)

My suggested compromise (or your tail-merging variant) is 100% backwards
compatible. People who don't want to take advantage of the fancy new
multi-terminal features will not be bothered with quirky
incompatibilities, while the value of DISPLAY will nicely follow
terminals.  I believe this was our primary mission now.  We can tweak
the details later, once everyone has had a chance to use the branch and
get acquainted with what it offers.

The new design can be basically explained in one short sentence:
"process-environment is terminal-local, but setenv affects all terminals."

* * *

This concludes the relevant discussion.  For the encore, I'd like to
keep beating some dead horses, and resolve some miscellaneous
misunderstandings:

> This is actually a good thing because it provides sort of a litmus
> test for interface usability and design quality: if one feels
> uncomfortable casting existing behavior into a clear description and
> instead is tempted to write something like "don't bother about it, it
> will magically do the right thing", then something is amiss.

I don't recall suggesting doing this or attempting to divert attention
like that.

For example, I believe "environment variables are client-local, but
setenv changes all clients at once" is a pretty useful description of
current environment semantics on the branch.  No handwaving is necessary.

>>      - For the user, there is a strong sense of connection between
>>      an emacsclient instance and its set of frames.  If emacsclient
>>      exits, then its frames are deleted and vice versa.
>>
>>      C-x C-c kills emacsclient, not the entire Emacs process.  All
>>      this feels very natural and fits a range of common use-cases,
>>      particularly the ones involving quick editing jobs from the
>>      command prompt.  (These are the ones for which Emacs was so
>>      infamously not well suited before.)
>>
>>        This means that having a different set of variables from frame to
>>        frame does not normally seem inconsistent or unpredictable to the
>>        user.
>
> I am opposed to claiming to create an illusion that we can't actually
> provide.

There is no illusion here.  There really is a strong sense of
connection.  The frame lives and dies with emacsclient.  This is quite
enough to the user to feel as if he had started a new editor instance.
We may choose to ignore this fact or we may try to take advantage of it,
but there is no denying it.

> kill-emacs should work as before.

It does work as before.  It kills Emacs. :-)

> However, one might
> consider providing a different command kill-terminal-group or similar
> which would instead be bound to C-x C-c.

That's exactly what the branch does.  The function is called
`save-buffers-kill-terminal'.

(My .emacs actually rebinds C-x C-c to a variant of the above that kills
Emacs itself when given a prefix argument.  Very useful to end the day.)

>>      - Furthermore, to me it seems more consistent to have all
>>      environment variables be local than just a select few of them.
>
> But it will pretty much break every Lisp program involved with the
> environment. That price is too high.

No it doesn't.  Having `process-environment' empty is what breaks _some_
of them.  Having all environment variables be in terminal-local bindings
in `process-environment' (i.e., "the new design") does not actually lead
to blatant breakage as such.

>>      - The behaviour of M-x shell and similar packages is mostly
>>      irrelevant here.
>
> Disagree.  "Similar packages" pretty much include _all_ packages that
> would have reason to access the environment, so _certainly_ those
> packages are relevant to the issue.

Um, nope.  M-x shell is special because it creates a long-term
subprocess with which the user may communicate with inside Emacs, from
different terminals.  X clients manually started from an M-x shell
buffer will appear on the terminal that was active at the time the shell
process was forked.  We can not change this fact, no matter how hard we
tweak Emacs's environment variable handling.  This is why I say the
behaviour of M-x shell, M-x term, GUD, ILISP and friends (a.k.a.
"similar packages") is irrelevant to this discussion.

Packages starting x clients such as xdvi and short subprocesses such as
compilations are, on the other hand, indeed relevant.

-- 
Karoly

signature.asc
Description: OpenPGP digital signature