Re: why does </proc/self/environ not work in bash?

[Christoph Anton Mitterer]

On Mon, 2024-02-12 at 10:06 +0900, Koichi Murase wrote:
> I quickly checked XCU 2.9.1, but it doesn't seem to specify when the
> fork (or in other words, the preparation of a separate execution
> environment) should be performed.

Well it says:
"3. Redirections shall be performed as described in Redirection"

Which is surrounded by stuff that still has to be done by the shell.

The next step:
"4. Each variable assignment shall be expanded for tilde expansion,
parameter expansion, command substitution, arithmetic expansion, and
quote removal prior to assigning the value."

While it may be variables that affect the executed program’s
environment, the expansions are clearly done by the shell, because the
executed program has no idea about that.

Also, all the following steps, about finding out what the command name
is, are clearly done in the context of the shell.


So independent of any optimisations, I would interpret this as
redirections should behave as if they were done by the shell.

And whether this is then done in a subshell, forked process, etc. pp.
I would at least expect the shell's environment to be present and the
command name be something that is not the program about to be executed.



>  The section "Command Search and
> Execution" mentions only execl.

But isn't that whole section already well beyond where POSIX says when
the redirections shall be performed?
Before that it even looks whether the name is a function, built-in etc.
- all which is clearly shell contect.


> From Item 2 for Variable assignments in XCU 2.9.1
> > If the command name is not a special built-in utility or function,
> > the variable assignments shall be exported for the execution
> > environment of the command and shall not affect the current
> > execution environment

But same here, the check whether the command name is a special built-in
or function, can only be made by the shell, not by the program to be
executed.
And that whole item comes well after "Redirections shall be performed
as described in Redirection."

2.7 also says
"A failure to open or create a file shall cause a redirection to fail."
which I'd interpret that at the "Redirection" step. the file is
actually opened/created (how could it fail, if it were not).


> As 2.9.1/(3) and 2.9.1/(4) may be swapped, ``the current execution
> environment'' for the redirections may be the execution environment
> of
> the command, though I'm not sure if this is the original intent of
> the
> standard or just a wording issue. Probably, it should be regarded as
> unspecified technically.

I would have anyway thought that the swapping clause doesn't apply to
our case, as in our case a command name *does* result, so the swapping
is *not* allowed.


> I think this is also an important point. The standards usually
> describe the behavior observed from the outside, and the way to
> realize the behavior is not restricted as far as the behavior is
> consistent. This is allowed for optimizations and workarounds for the
> implementation issues. This means that if there is a way to take a
> peek inside the implementation, there can be something in
> intermediate
> steps that doesn't follow the exact processing steps POSIX specifies.
> Procfs is not a part of POSIX but is closely related to the system.
> Once you use it, you cannot expect the behaviors that POSIX
> specifies.

As I've said... though same time OTOH, if one follows the order given
by 2.9.1. then in my understanding the standard would say, that (in our
case) the redirections (with actually opening the files) should happen
at step 3, and at least from my arguing above I'd say that's
conceptually still considered to be done within/by the shell.

Then, even if the /proc/ files are special,.. they'd still have a well
defined state at that point.


> You might think the behavior of dash is the desired one, but dash
> also
> doesn't always behave as you expect (This point was also mentioned by
> Kerin and in the wiki page pointed by Lawrence).

Well let me make clear that this is not about playing off bash against
dash. :-)

I merely noticed that behaviour by accident when trying to make my
little function that checks the behaviour of a shell regarding
exportation of env vars that have invalid shell var names... and found
the behaviour to be not what I'd have naively expected.


> Yash is the only one in this context. If you think there is still a
> conflict with the standard, I think what should be fixed is not the
> shells but the standard. The standard needs to make it explicit that
> it is unspecified if you believe the current wording is not enough.

Would in principle also be fine.

Though, admittedly, I'd rather say that in this particular case, bash's
behaviour is not what one would expect, and a change there might also
be justified.

> 
> See this reply:
> 
> https://lists.gnu.org/archive/html/help-bash/2024-02/msg00026.html

What I meant is not:
How does it technically happen, that the env is empty.

I rather meant:
Why would that be allowed by POSIX (or be reasonably expected by
users)?

It's fine if implementations do optimisations, but it doesn't seem to
me that e.g. 2.9.1 would reasonably allow that at some step in there
the environment of the shell is suddenly gone.

Though admittedly in this specific case, we really are in /proc-
territory which is unchartered by POSIX.


> 
> I still don't understand why you need /proc/self/environ. In the
> above
> case you can just write env -i 'VAR=...' '+=...' "/proc/$$/exe" -c
> "..."

Uhm, I merely didn't think to end at first, that env (without
arguments) also prints the env vars with names that are invalid shell
var names.
That's why within the -c command string, I did something like cat
/proc/self/environ, to get the env of a program, started by the
"/proc/$$/exe" shell.

But even that wouldn't suffer form all the above, as there were no
redirections involved and it would have been clearly cat which gets
/proc/self/environ as an argument and opens it.

But sure, simply using env twice is much easier and ships around any
problems involved with /proc vs. POSIX.


Cheers,
Chris.