Re: Improving Shepherd

[Jelle Licht]

Hey all,

2018-02-05 14:08 GMT+01:00 Ludovic Courtès <address@hidden>:

Hello!

[...]

Currently shepherd monitors SIGCHLD, and it’s not supposed to miss
those; in some cases it might handle them later than you’d expect, which
means that in the meantime you see a zombie process, but otherwise it
seems to work.

ISTR you reported an issue when using ‘shepherd --daemonize’, right?
Perhaps the issue is limited to that mode?

Playing around with signalfd(2) for a bit, it seems that implementations are
allowed to coalesce several 'pending' signals at the same time. In the case
of SIGCHLD, this means the parent process might never be properly
informed of *mutliple* signals being received around the same time. Could

it have something to do with this problem as well?

> Concurrency/parallelism - I think Jelle was planning to work on this,
> but I might be wrong about that. Maybe I volunteered? We're keen to
> see Shepherd starting services in parallel, where possible. This will
> require some changes to the way we start/stop services (because at the
> moment we just send a "start" signal to a single service to start it,
> which makes it hard to be parallel), and will require us to actually
> build some sort of real dependency resolution. Longer-term our goal
> should be to bring fibers into Shepherd, but Efraim mentioned that
> fibers doesn't compile on ARM at the moment, so we'll have to get that
> working first at least.

I’d really like to see that happen.  I’ve become more familiar with
Fibers, and I think it’ll be perfect for the Shepherd (and we’ll fix the
ARM build issue, no doubt.)

One thing I’d like to do is to handle SIGCHLD via signalfd(2) instead of
an actual signal handler like we do now.  That would make it easy to
have signal handling part of the main event loop and thus, it would
integrate well with Fibers.

It seems that signalfd(2) is Linux-only though, which is a bummer.  The
solution might be to get over it and have it implemented on GNU/Hurd…
(I saw this discussion:
<https://www.gnu.org/software/hurd/glibc/signal/signal_thread.html>; I
suspect it’s within reach.)

Good news: signfalfd seems to work as far as I can see. I am not quite sure
how to make it work consistently with guile ports yet though.

To make use of signalfd, one normally masks signals so that these can
handled via signalfd instead of the default signal handlers; any process
forked start out with the same signal mask, so we would need to make
sure to either reset the signal mask for spawned processes.

[...]

Ludo’.

Jelle