Re: [PATCH] Unix domain socket filename redirection

[tetsujin]

Unix Domain Sockets form the basis of a lot of the modern forms of
Unix/Posix IPC. Compared to using TCP sockets for IPC, Unix Domain
sockets allow you to control access to the resource with filesystem
permissions, and identify the socket with a file path so you don't
have to come up with a system-wide-unique port number for the service.
It's kind of like using named pipes created with mkfifo(), except they
can accept multiple connections, and you can transfer not only data
values over the channel, but open file descriptors and system
privileges as well.

So implementing Unix Domain Sockets in the shell would give you the
ability to connect to existing services on the machine that use named
sockets for control (systemd, dbus, cups, etc.) or you could use the
feature to implement new services and connect to them with shell
scripts, or just use them to connect different scripts and programs
together for various forms of IPC.

(As a side note, the shell somewhat defines the user's operating
environment. I think that not providing a feature because of a
perception that "no one uses it" becomes somewhat self-fulfilling: No
one writes shell-friendly Unix Domain Socket servers because the shell
doesn't support connecting to them, and the definition of what's
"shell-friendly" is rather narrow.)

I mentioned this in an earlier post, but I have written an
implementation of socketpair, sendmsg, and recvmsg as shell loadable
built-ins, in a lib I call "shell-pepper". This makes it possible to
implement new "open a file descriptor" commands as external
executables rather than as shell built-ins:

$ socketpair --cloexec s    # Creates a bidirectional Unix Domain
communication channel, stores the file descriptors for the two ends in
${s[@]}.
$ connect_to_socketfile ./socket_to_me >&${s[1]} &   #
"connect_to_socketfile" is an external command that connects to a
socket and sends it over FD 1 using sendmsg() and SCM_RIGHTS
$ recvmsg message connection_fd <&${s[0]}   # Receives the open file
descriptor send by connect_to_named_socket, assigns it an unused file
descriptor, and stores that FD in $connection_fd

I think this speaks to the benefits of building Unix Domain Socket
functionality into the shell: it allows adding other functionality
like this without having to tie it directly into the shell process.

As for the redirection syntax, it's useful because the shell manages
the lifetime of file redirections, like this:

$ cmd <>/dev/unixstream/./smelly_sock   # Socket "./smelly_sock" is
opened, used for the I/O of "cmd", and then closed by the shell
afterward.

As opposed to something like this:

$ connect_to_socketfile ./smelly_sock connection_fd    # Assuming
"connect_to_socketfile" were a built-in, rather than an external
command as it is described above
$ cmd <>&$connection_fd
$ exec {connection_fd}<&-    # Explicitly close the file descriptor

Though even with the benefits of redirection, personally I oppose the
idea of spoofing more fake devices in /dev. (The shell spoofs some
stuff in /dev already, and it's all long-established at this
point...  But I don't think more such fake devices should be added.)
In this patch, spoofing /dev grants us the ability to specify a single
socket option. It's not a great way to set options, though. What if
the API changed, and we suddenly wanted to be able to specify two
options? Specifying options as command arguments is both clearer and
more flexible.

I've given some thought to how the two approaches could be combined:
use a command to open a file descriptor, but bind the file
descriptor's lifetime to a redirection. For instance, consider this
concept for a new redirection syntax:

$ cmd <>&(connect_to_socketfile --datagram ./rockem_sockem)   #
Connect to socketfile, bind it as stdin and stdout for running "cmd",
then undo the redirection and close the socket

In this hypothetical model, the redirection syntax <&(fd_cmd) runs
"fd_cmd" with its output redirected to one end of a socketpair.
"connect_to_socketfile" creates the connection then tries to send it
over file descriptor 1 using sendmsg() and SCM_RIGHTS (see "man 7
unix")
 The shell process listens to the other end of the socketpair,
expecting to receive a file descriptor. If it does receive a file
descriptor, it uses that FD for the redirection, and then disposes of
it when the job is done.
(And if "connect_to_socketfile" returns nonzero status, or doesn't
send exactly one file descriptor over its output socket, then the
redirection fails and the command is aborted.)

That said, there are always problems with introducing new syntax... 
I don't know if this specific syntax would be viable (and it's hard to
come up with syntax that seems conceptually coherent with other forms
of redirection and substitution) but I think something like this could
be useful as a way to add new functionality like this to the shell,
and take advantage of redirection semantics, without having to
integrate the implementation of each new feature into the shell.

And it's worth noting that while Unix Domain Socket functionality is
widespread, established as part of POSIX 2001 and all that, it's not
universally-supported on all (especially older) platforms Bash
supports. So even if it were integrated into the shell, it still must
be considered an optional feature. It may be possible to provide a
single interface that would also cover other APIs with similar
features (the STREAMS API maybe?)

----- Original Message -----
From: 
To:"Z" <zartaj@gmail.com>
Cc:<bug-bash@gnu.org>
Sent:Mon, 19 Jun 2017 22:36:40 -0500
Subject:Re: [PATCH] Unix domain socket filename redirection

 On Mon, Jun 19, 2017 at 07:30:42PM -0400, Z wrote:
 > This patch adds support for Unix domain sockets. A filename of the
form
 > /dev/unixstream/path used in a redirection makes bash attempt to
open a
 > connection to the Unix domain stream socket at /path. Similarly a
filename
 > of the form /dev/unixdgram/path makes bash attempt to open a
connection to
 > the Unix domain datagram socket at /path.

 Excuse my ignorance:

 - What would be an expected use case of this feature? /dev/tcp and
 /dev/udp are commonly used because it allows the shell script writer
 to interact with popular text based protocols like HTTP or SMTP. It
 would seem that this UNIX domain socket patch targets a niche
instead?

 - Why should this be integrated into the core of bash, and not
provided
 as a loadable builtin instead? It makes more sense to me to provide
 this as an optional shell builtin, which also make the usage less
 awkward (not a fan of the /dev/unixstream/the/path/is/here thing).

 By the way, the patch does not apply. It would seem like you
copy-pasted
 the diff, so it got wrapped at some points and git is rejecting it.

 -- 
 Eduardo Bustamante
 https://dualbus.me/