Re: socket.c added support for unix domain socket datagram transport

[Stefano Brivio]

Hi Ralph,

On Fri, 23 Apr 2021 08:56:48 +0200
Ralph Schmieder <ralph.schmieder@gmail.com> wrote:

> Hey...  new to this list.  I was looking for a way to use Unix domain
> sockets as a network transport between local VMs.
> 
> I'm part of a team where we run dozens if not hundreds of VMs on a
> single compute instance which are highly interconnected.
> 
> In the current implementation, I use UDP sockets (e.g. something like 
> 
> -netdev
> id=bla,type=socket,udp=localhost:1234,localaddr=localhost:5678) 
> 
> which works great.
> 
> The downside of this approach is that I need to keep track of all the
> UDP ports in use and that there's a potential for clashes.  Clearly,
> having Unix domain sockets where I could store the sockets in the
> VM's directory would be much easier to manage.
> 
> However, even though there is some AF_UNIX support in net/socket.c,
> it's
> 
> - not configurable
> - it doesn't work

I hate to say this, but I've been working on something very similar
just in the past days, with the notable difference that I'm using
stream-oriented AF_UNIX sockets instead of datagram-oriented.

I have a similar use case, and after some experiments I picked a
stream-oriented socket over datagram-oriented because:

- overhead appears to be the same

- message boundaries make little sense here -- you already have a
  32-bit vnet header with the message size defining the message
  boundaries

- datagram-oriented AF_UNIX sockets are actually reliable and there's
  no packet reordering on Linux, but this is not "formally" guaranteed

- it's helpful for me to know when a qemu instance disconnects for any
  reason

> As a side note, I tried to pass in an already open FD, but that
> didn't work either.

This actually worked for me as a quick work-around, either with:
        https://github.com/StevenVanAcker/udstools

or with a trivial C implementation of that, that does essentially:

        fd = strtol(argv[1], NULL, 0);
        if (fd < 3 || fd > INT_MAX || errno)
                usage(argv[0]);

        s = socket(AF_UNIX, SOCK_STREAM, 0);
        if (s < 0) {
                perror("socket");
                exit(EXIT_FAILURE);
        }

        if (connect(s, (const struct sockaddr *)&addr, sizeof(addr)) < 0) {
                perror("connect");
                exit(EXIT_FAILURE);
        }

        if (dup2(s, (int)fd) < 0) {
                perror("dup");
                exit(EXIT_FAILURE);
        }

        close(s);

        execvp(argv[2], argv + 2);
        perror("execvp");

where argv[1] is the socket number you pass in the qemu command line
(-net socket,fd=X) and argv[2] is the path to qemu.

> So, I added some code which does work for me... e.g.
> 
> - can specify the socket paths like -netdev
> id=bla,type=socket,unix=/tmp/in:/tmp/out
> - it does forward packets between two Qemu instances running
> back-to-back
> 
> I'm wondering if this is of interest for the wider community and, if
> so, how to proceed.
> 
> Thanks,
> -ralph
> 
> Commit
> https://github.com/rschmied/qemu/commit/73f02114e718ec898c7cd8e855d0d5d5d7abe362

I think your patch could be a bit simpler, as you could mostly reuse
net_socket_udp_init() for your initialisation, and perhaps rename it to
net_socket_dgram_init().

Anyway, I wonder, would my approach work for you instead? I'm posting my
patch in a minute.

-- 
Stefano