[Gnu-arch-users] filename issues (was Re: [...] naming scheme is shell unfriendly)

[Thomas Lord]

  > The current arch's naming scheme is pretty inconvenient. You may
  > get used to it, but you can't claim it is shell friendly (whether
  > your shell is bash, tcsh or zsh). This is not to deny that in a
  > fantazy world with arch and no other citizens, such naming scheme
  > would possibly rock.

Maybe I'm simply a more effective Unix user than many people.   So,
here are some hints and after that some gossipy history, and after
that, a tiny design lesson.


* Clues About `Shells+Shell-Tools v. Arch'

  My primary shell is Emacs (leveraging $SHELL to launch command lines
  assembled and editted entirely within Emacs), enhanced by a tiny
  customization package called `background.el' that has been circulated
  informally for something between 15 and 20 years now.  Quite possibly
  the shell commands that currently come installed with Emacs can
  finally do what `background.el' does --- I'm not sure.

  My secondary shells are bash or sh, run in an emacs buffer,
  where `readline' features built into a shell (tab completion,
  etc.) are useless:  Emacs handles all completion for me.

  My tertiary shells are any sh-varient, run on a normal tty.  I rarely
  need to run more than one or a few commands this way.

  I have customized my Emacs set-up so that I have not one but two
  distinct "completion keys" (like TAB with bash on a normal terminal).
  One completion key runs a general purpose completer (called
  "hippie-expand") which follows lots of context-specific heuristics to
  pick an expansion.   The other completion key runs
  "dynamic-complete-filename".  I rarely have any problem completing
  filenames accurately.

  One very nice property of this set-up: all of my completion tools 
  work regardless of what program is reading input.   For example, even
  when I use an FTP client that lacks `readline' features my
  completion keys work just fine because it is a front-end program
  (Emacs) which provides them.

  In theory, I should have more problems with file globbing (e.g.,
  trying to list a comma-file in a "{...}" glob).  In practice I don't
  because, almost always, instead of globbing features, I use a facile,
  quick, outline-structured directory-tree browser to select files
  (using regexps, regexps for file-name extensions, file types, etc.)
  and pass the selection to a shell command.  (I use my own `monkey.el'
  which could use a good thorough rewrite but which continues to work,
  nonetheless.)

  So for me, the vast majority of the problems people have about
  file-names with punctuation simply never occur.  Arch lives in a world
  with every other program just fine and it all works swimmingly (with
  some exceptions, see below).

  Many other people work with similar setups.  I seem to recall that
  such a setup can be pretty well approximated on a normal tty using
  programs such as `screen(1)' and so the functionality I'm describing
  is not out of reach even to those who loathe the idea of learning
  emacs.

  In practice, I /do/ have some problems with filename completion,
  although nothing among these problems is specific to arch.   The
  single, largest (by far) specific problem I have is the GNU convention
  for specifying arguments to long-named options:

  [[
        --prefix=/usr/scratch
  ]]

  The mandatory `=' in that syntax breaks the usual rule that a filename
  completer can scan backwards for whitespace and count everything back
  to that point as filename.

  That's why, when I wrote the option parser for package-framework, 
  I made sure to add support for an alternative syntax:

  [[
        --prefix /usr/scratch
  ]]

  Sadly, when an author of Bash saw similar problems with `=', rather
  than working on fixing the broken command line interfaces that caused
  it, he chose to add a special case to file-name completion in the
  `readline' part of bash.   It was a giddy time: people did lots of
  crazy things back then.

  Sometimes, by the way, I get to use a version of /bin/sh which, unlike
  the implementations mentioned in the quote above, was developed and is
  maintained quite conservatively.   No `readline' features such as tab
  completion are provided and the globbing features and string
  interpolation features are kept very close to original tradition and
  standard.   These shells never give me any problem, either, even on a
  dumb tty -- although yes, they do require typing things out.

  Another problem I have witnessed once or twice but can't say I run
  into in practice is the problem of programs whose CLI option and
  argument syntax is flagrantly non-conventional.  Some versions of
  `more', `vi' -- these programs can easily flip out about file names
  that begin with `+'.



* Gossipy History

  The hackers of yore scoffed when the `readline' library appeared in
  `bash'.   ``Aside from making for a bloated shell," they said, "you're
  not respecting the /predictable/ simplicity of the traditional shell.
  In essense, you're just abandoning the ``tiny tools'' approach to 
  writing software systems.''   These fabled hackers would have said
  similar things about zsh's exhuberent extension of sh meta-syntax,
  too, I think.

  The newbie-hackers of yore names of the free software world
  (currently the upper-middle generation of hackers), considered those
  ur-hackers to be neo-luddites.  Roughly, the reasoning was:
  ``Tops-10 and Tops-20 and VMS have all these cool completion
  features and whatnot.  New and experienced users really like them
  and look, I made an approximation of them.  So, nyah-nyah.''

  To twist the old cliche:  it's amazing how much smarter the older
  hackers got while the younger ones were learning.

  Bash (and GNU utils generally) started out with a noble goal and a
  huge, vast, amazing opportunity to achieve that goal: then blew it,
  big time.

  The noble goal: implement DEC-style completion and on-line help but 
  do it on unix this time.   And, oh yeah, keep upward compatability
  with traditional unix.

  Blowing it: the tactic of ``stick `readline' in every package''.
  This was predictably dumb and the above discussion refers to many
  unfortunate (but reasonably predictable at the time) consequences of
  this move.

  How could they not have blown it? ...


* Design Thoughts

** the cli is an api

  The CLI is an API.   In C, the language in which shell APIs is
  designed is layered up the interface to `execve'.  On top of the
  execve API, CLIs add syntax for argv, usually including some notion
  of /typed/ arguments.

  Sadly, few applications use a common language to layer their CLI
  on top of `execve'.   The CLI syntax of `vi' options violates the
  hoped-for general rule that option names begin with `+', for
  example.   And there is no common "type system" for CLI options
  and arguments.

  Stopping with `argc/argv' as the highest level of abstraction for
  CLIs made a lot of sense on a sluggish (by today's standards) PDP-11
  with 64K process address spaces and, if you were lucky, entire 10s
  of megs of disk storage, in 1982.


** the cli ought to be reflective

  Fancy on-line, context-sensative help and completion systems can be
  great.   Why do they need to be built into every program?

  Rather, since both help and the marshalling of command arguments can
  be accomplished so many different ways (e.g., typing a line to a shell,
  issuing an emacs command in a directory editor, clicking `do-it' in
  a gui) -- surely what is needed is not help and completion /code/
  //in// every application but rather help-supporting and
  completion-supporting /data/ available //about// every application.

  I've long commented about how help messages in arch eventually need
  to be cleaned up.  They are fairly regular already but, ideally,
  they will be machine parsable (or available via an alternative
  interface in machine parsable format) sufficiently that, for
  example, an effective and feature-rich dialog box can be
  automatically constructed for each command given only the data
  available in its `--help' output.


** armed with a reflective capability on primitive commands,
   interfaces sing

  For example:

  @@

  ~ {http://segusoland.sourceforge.net} 

    The "reciprocal narrowing" GUI hack.

  ~ 
{http://www.gnu.org/software/emacs/elisp-manual/html_node/elisp_288.html#SEC288}

    An entry point to gaining an understand of why Emacs
    and the ``tiny tools'' approach to software development
    are, in the end, pretty much the same thing.

  @@

-t