Re: Improving Shepherd

[Ludovic Courtès]

Carlo Zancanaro <address@hidden> skribis:

> Hey Ludo,
>
> On Mon, Feb 05 2018, Ludovic Courtès wrote:
>>> User services - Alex has already sent a patch to the list to allow
>>> generating user services from the Guix side. The idea is to
>>> generate a
>>> Shepherd config file, allowing a user to invoke shepherd manually
>>> to
>>> start their services. A further extension to this would be to have
>>> something like systemd's "user sessions", where the pid 1 Shepherd
>>> automatically starts a user's services when they log in.
>>
>> After replying to Alex’ message, I realized that we could just as
>> well
>> have a separate “guix service” or similar tool to take care of this?
>>
>> This needs more thought (and perhaps taking a look at systemd user
>> sessions, which I’m not familiar with), but I think Alex’ approach
>> is a
>> good starting point.
>
> We were thinking it might work like this:
> - services->package constructs a package which places a file in the
> profile containing the necessary references
> - pid 1 shepherd listens to elogind login/logout events, and starts
> the services when necessary
>
> Admittedly this isn't the nicest way for it to work, but it might be a
> good starting point.

Yes, sounds reasonable.

> There were some discussions on the list a while ago about how to have
> `guix environment` automatically start services, too, so I wonder what
> overlap there could be there. Although maybe environment services (in
> containers) have more in common with system services than user
> services.

That’s a separate topic I think, but I agree it’d be useful.

>> Currently shepherd monitors SIGCHLD, and it’s not supposed to miss
>> those; in some cases it might handle them later than you’d expect,
>> which
>> means that in the meantime you see a zombie process, but otherwise
>> it
>> seems to work.
>>
>> ISTR you reported an issue when using ‘shepherd --daemonize’, right?
>> Perhaps the issue is limited to that mode?
>
> I no longer use the daemonize function. My user shepherd runs "in the
> foreground" (it's started when my X session starts), so it's not
> that. Jelle fixed the problem I was having by delaying the SIGCHLD
> handler registration until it's needed. It is still buggy if a process
> is started before the daemonize command is given to root service,
> though.
>
> If you try running "emacs --daemon" with "make-forkexec-constructor"
> (and #:pid-file, and put something in your emacs config to make it
> write out the pid) you should be able to reproduce what I am
> seeing. If you kill emacs (or if it crashes) then shepherd continues
> to report that it is started and running. When I look at htop's output
> I can also see that my emacs process is not a child of my shepherd
> process.
>
> I would like to add a --daemon/--daemonize command line argument to
> shepherd instead of the current "send the root service a daemonize
> message". I think the use cases of turning it into a daemon later are
> limited, and it just gives you an additional way of shooting yourself
> in the foot.

Also a separate topic ;-), but if you still experience a bug, please
report it and see whether you can provide a reduced test case to
reproduce it.

>> I’d really like to see that happen.  I’ve become more familiar with
>> Fibers, and I think it’ll be perfect for the Shepherd (and we’ll fix
>> the
>> ARM build issue, no doubt.)
>
> I'm not going to put much time/effort into this until we have fibers
> building on ARM.

Hopefully it’s nothing serious: Fibers doesn’t rely on anything
architecture-specific.

> I think these changes are likely to break shepherd's config API,
> too.

I’m not sure.  We may be able to keep the exact same API.  At least
that’s what I had in mind for the first Fibers-enabled Shepherd.

> In particular, with higher levels of concurrency I want to move the
> mutable state out of <service> objects.

The only piece of mutable state is the ‘running’ value.  We can make
that an “atomic box”, and users won’t even notice.

>> It seems that signalfd(2) is Linux-only though, which is a bummer.
>> The
>> solution might be to get over it and have it implemented on
>> GNU/Hurd…
>> (I saw this discussion:
>> <https://www.gnu.org/software/hurd/glibc/signal/signal_thread.html>;
>> I
>> suspect it’s within reach.)
>
> Failing that, could we have our signal handlers just convert the
> signal to a message in our event loop?

Yes, they could send a message on a Fibers channel.

Thanks,
Ludo’.