Re: posix command search and execution

[Robert Elz]

    Date:        Mon, 6 Nov 2023 14:28:24 -0500
    From:        Chet Ramey <chet.ramey@case.edu>
    Message-ID:  <0ab6075e-22bf-43cd-992c-b2476f62629f@case.edu>

  | On 11/6/23 10:48 AM, Mike Jonkmans wrote:
  | > According to these docs (what I make of it), resolving is done
  | > in steps, the first applicable step is used:

This is one of the most debated, and stupidest, parts of posix.

  | > 1b) List several names that have unspecified results.
  | This is an ad-hoc list of builtins that shells implement,
  | not necessarily common across all shells.

If it were just builtins it would not be important, the issue
is more that some shells implement some of that list as reserved
words, or aliases, and if that's done what applications can do
alters dramatically.   So avoiding using those words as command
names, except when using the known features of a specific shell,
is the best way to remain portable.

  | > 1c) Use a function, for functions not matching standard utilities.

No, that's not what it says, it is except of standard utilities
implemented as functions.   More on that below.

  | > 1d) Lists 20 fixed utility names (like alias, cd etc.) that are
  | >      to be invoked at his point. No PATH search yet.
  | >   These are the `regular builtins'.

In the next standard the ones listed are the intrinsic builtins,
and includes only those that must be builtin to work.   But
implementations can add more to the list.

  | >   (These need not exist as builtin).
  | These are the historical common builtins.

That is how the existing standard is written.

  | > 1eI) Search is successful.
  | > 1eIa) Check for `regular builtins' and functions
  | >        and invoke that regular builtin/function.
  | >        Q: Shouldn't this specify an ordering for builtins/functions?
  |
  | The text seems to imply that you can't have both, doesn't it?

While I suppose you could have both, it would be very unusual.
Again, the functions that can get invoked here are only the
standard utilities implemented as functions, all others would
have been invoked earlier, and we would never be here.

That phrasing is meant to apply to a standard utility (which
just means any utility defined by the standard, as distinct
from others added be the implementation, or user) that is
implemented as a function, by the implementation.  It is
not meant to apply to a function that the application happens
to have defined with the same name as a standard utility.

  | My feeling, without testing anything, is that most shells would allow
  | functions to override builtins here.

Since I have never seen any shell implement any standard utility
as a function, it would be very hard to test.   Further if the
did, also implementing the same thing as a builtin would be
even harder to imagine - why do it twice when one of the two
would never be used?   So not just hard to test - probably
impossible.

It is also unclear to me why anyone would ever implement a standard
builtin as a function - implementing builtins is simpler for the
implementation than functions (in my experience anyway) and in
any case, if the rules in the standard are followed, there is
no way (except possibly by using "command", and even that is not
clear to me) to tell if the implementation used a function or
a builtin (maybe the output from type might make it clear, but
not necessarily).

  | This has been an area of significant disagreement.

It has indeed.

  | > 1eIb) Run the utility.
  | >        (This is where ordinary builtins should run).
  | >     (It seems logical that a builtin takes precedence over PATH).
  |
  | You'd be surprised.

Yes.   But almost all shells implement it that way, so the
seemingly logical assumption is mostly backed by experience.

  | Note that this seems to require that you can only run
  | a builtin if it exists (or something with that name exists) in $PATH.

A builtin for a standard utility, yes.  Unless the implementation has
defined it as intrinsic (which the forthcoming standard allows, but
discourages).  Applications (which includes users) who invoke non
standard utilities are stepping outside the standard, so get
unspecified results (so implementations can add new non-standard
builtins without also adding a matching command in PATH without
issues.

  | So if you have a builtin that doesn't exist in $PATH and isn't listed as
  | one of the regular builtins, what do you do? Even the unspecified list
  | doesn't give much help.

If it is a standard utility it is required to exist in PATH.
If PATH has been changed so that is no longer true, then that
is a non-conforming environment, and anything is OK.  Similarly
if the builtin is not a standard utility (like declare or
enable for example).

  | This is a quality of implementation feature.
  | Why confuse users by allowing them to define a function that
  | will never be executed?

Indeed - but you could also write that as "Why confuse users by
allowing them to define a function that can never be invoked?"
and by so doing, encourage more portable scripts.

In practice this distinction (unlike some of the other properties
os special builtins) rarely matters, as users typically have no
reason to define functions that override the special builtins.

  | I think the resolution to interpretation 854 addresses this. Shells
  | who want this ordering just declare all the builtins they implement as
  | `intrinsic' so they're not subject to a PATH search.

Yes.

  | Kind of; regular builtins aren't really defined anywhere.

They are, kind of (look in XBD) - they are just any utility
the implementation happens to have implemented inside the shell.
Aside from execution speed, you're not supposed to be able to
tell the difference.   That's the inspiration for the PATH
searching nonsense.  Or seems to be - but that isn't really
correct, as if a user was to install their own version of
a standard utility in PATH before the normal one (which is
what all this is supposed to allow) then either that version
implements the standard, in which case a user using only standard
features still cannot tell the difference between that one and
the builtin (speed ignored) or the standard no longer applies,
and the results (including which gets invoked) are unspecified
anyway.

  | >    Q: Where is `standard utilities' defined - as used in 1d.
  | These are the standard utilities.

I am not sure what "these" was intended to refer to, but standard
utilities are the set of all the utilities defined in the standard.

kre