[shepherd] 02/04: Add resource monitoring service.

[Ludovic Courtès]

civodul pushed a commit to branch master
in repository shepherd.

commit c64804f3a95f0fc74bcf75cd76975f67817f797d
Author: Ludovic Courtès <ludo@gnu.org>
AuthorDate: Sat Mar 11 19:58:29 2023 +0100

    Add resource monitoring service.
    
    * modules/shepherd/service/monitoring.scm: New file.
    * modules/shepherd/service.scm (get-message*): Export.
    * tests/services/monitoring.sh: New file.
    * doc/shepherd.texi (Service Collection): New chapter.
    * po/POTFILES.in: Add 'monitoring.scm'.
    * Makefile.am (servicesubdir, servicegosubdir)
    (dist_servicesub_DATA, nodist_servicegosub_DATA): New variables.
    (TESTS): Add 'tests/services/monitoring.sh'.
---
 Makefile.am                             |   9 ++-
 doc/shepherd.texi                       |  77 ++++++++++++++++++++++--
 modules/shepherd/service.scm            |   4 +-
 modules/shepherd/service/monitoring.scm | 100 ++++++++++++++++++++++++++++++++
 po/POTFILES.in                          |   1 +
 tests/services/monitoring.sh            |  62 ++++++++++++++++++++
 6 files changed, 245 insertions(+), 8 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index 7c57ff8..be0069b 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -36,6 +36,7 @@ dist_guilemodule_DATA =                               \
 nodist_guileobject_DATA = $(dist_guilemodule_DATA:%.scm=%.go)
 
 shepherdsubdir = $(guilemoduledir)/shepherd
+servicesubdir = $(guilemoduledir)/shepherd/service
 dist_shepherdsub_DATA =                                \
   modules/shepherd/args.scm                    \
   modules/shepherd/service.scm                 \
@@ -44,12 +45,17 @@ dist_shepherdsub_DATA =                             \
 nodist_shepherdsub_DATA =                      \
   modules/shepherd/config.scm                  \
   modules/shepherd/system.scm
+dist_servicesub_DATA =                         \
+  modules/shepherd/service/monitoring.scm
 
 shepherdgosubdir = $(guileobjectdir)/shepherd
+servicegosubdir = $(guileobjectdir)/shepherd/service
 nodist_shepherdgosub_DATA =                    \
   modules/shepherd/config.go                   \
   modules/shepherd/system.go                   \
   $(dist_shepherdsub_DATA:%.scm=%.go)
+nodist_servicegosub_DATA =                     \
+  $(dist_servicesub_DATA:%.scm=%.go)
 
 scriptsdir = $(shepherdsubdir)/scripts
 dist_scripts_DATA =                            \
@@ -253,7 +259,8 @@ TESTS =                                             \
   tests/systemd.sh                             \
   tests/signals.sh                             \
   tests/system-star.sh                         \
-  tests/close-on-exec.sh
+  tests/close-on-exec.sh                       \
+  tests/services/monitoring.sh
 
 TEST_EXTENSIONS = .sh
 EXTRA_DIST += $(TESTS)
diff --git a/doc/shepherd.texi b/doc/shepherd.texi
index 431e4b3..6fdf7c1 100644
--- a/doc/shepherd.texi
+++ b/doc/shepherd.texi
@@ -58,12 +58,13 @@ This manual documents the GNU@tie{}Shepherd version 
@value{VERSION}, a
 service manager for the GNU system.
 
 @menu
-* Introduction::      Introduction to the Shepherd service manager.
-* Jump Start::        How to do simple things with the Shepherd.
-* herd and shepherd:: User interface to service management.
-* Services::          Details on services.
-* Misc Facilities::   Generally useful things provided by the Shepherd.
-* Internals::         Hacking shepherd.
+* Introduction::                Introduction to the Shepherd service manager.
+* Jump Start::                  How to do simple things with the Shepherd.
+* herd and shepherd::           User interface to service management.
+* Services::                    Details on services.
+* Service Collection::          Services that come with the Shepherd.
+* Misc Facilities::             Generally useful things provided by the 
Shepherd.
+* Internals::                   Hacking shepherd.
 
 * GNU Free Documentation License::  The license of this manual.
 * Concept Index::
@@ -1415,6 +1416,70 @@ respawnable services are started, as otherwise we would 
not get the
 
 @end table
 
+
+@c @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
+
+@node Service Collection
+@chapter Service Collection
+
+The Shepherd comes with a collection of services that let you control it
+or otherwise extend its functionality.  This chapter documents them.
+
+@menu
+* Monitoring Service::          Monitoring shepherd resource usage.
+@end menu
+
+@node Monitoring Service
+@section Monitoring Service
+
+The @dfn{monitoring service}, as its name suggests, monitors resource
+usage of the shepherd daemon.  It does so by periodically logging
+information about key resources: heap size (memory usage), open file
+descriptors, and so on.  It is a simple and useful way to check whether
+resource usage remains under control.
+
+To use it, a simple configuration file that uses this service and
+nothing else would look like this:
+
+@lisp
+(use-modules (shepherd service monitoring))
+
+(register-services
+  ;; Create a monitoring service that logs every 15 minutes.
+  (monitoring-service #:period (* 15 60)))
+
+;; Start it!
+(start 'monitoring)
+@end lisp
+
+Using the @code{herd} command, you can get immediate resource usage
+logging:
+
+@example
+$ herd log monitoring
+heap: 8.77 MiB; file descriptors: 20
+@end example
+
+You can also change the logging period; for instance, here is how you'd
+change it to 30 minutes:
+
+@example
+$ herd period monitoring 30
+@end example
+
+The @code{(shepherd service monitoring)} module exports the following
+bindings:
+
+@deffn {procedure} monitoring-service @
+          [#:period (default-monitoring-period)]
+Return a service that will monitor shepherd resource usage by printing it
+every @var{period} seconds.
+@end deffn
+
+@defvr {Scheme Variable} default-monitoring-period
+This parameter specifies the default monitoring period, in seconds.
+@end defvr
+
 @c @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
 
 @node Misc Facilities
diff --git a/modules/shepherd/service.scm b/modules/shepherd/service.scm
index 4562743..15e1a12 100644
--- a/modules/shepherd/service.scm
+++ b/modules/shepherd/service.scm
@@ -139,7 +139,9 @@
             action-runtime-error-key
             action-runtime-error-arguments
 
-            condition->sexp))
+            condition->sexp
+
+            get-message*))                    ;XXX: for lack of a better place
 
 
 (define sleep (@ (fibers) sleep))
diff --git a/modules/shepherd/service/monitoring.scm 
b/modules/shepherd/service/monitoring.scm
new file mode 100644
index 0000000..338c08d
--- /dev/null
+++ b/modules/shepherd/service/monitoring.scm
@@ -0,0 +1,100 @@
+;; monitor.scm -- Monitoring service.
+;; Copyright (C) 2023 Ludovic Courtès <ludo@gnu.org>
+;;
+;; This file is part of the GNU Shepherd.
+;;
+;; The GNU Shepherd is free software; you can redistribute it and/or modify it
+;; under the terms of the GNU General Public License as published by
+;; the Free Software Foundation; either version 3 of the License, or (at
+;; your option) any later version.
+;;
+;; The GNU Shepherd is distributed in the hope that it will be useful, but
+;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with the GNU Shepherd.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (shepherd service monitoring)
+  #:use-module (shepherd service)
+  #:use-module (shepherd support)
+  #:use-module ((fibers) #:hide (sleep))
+  #:use-module (fibers channels)
+  #:use-module (ice-9 match)
+  #:autoload   (ice-9 ftw) (scandir)
+  #:use-module (oop goops)
+  #:export (monitoring-service))
+
+(define (log-monitoring-stats)
+  "Log info about useful metrics: heap size, open file descriptors, etc."
+  (local-output (l10n "heap: ~,2f MiB; file descriptors: ~a")
+                (/ (assoc-ref (gc-stats) 'heap-size) (expt 2. 20))
+                (length
+                 (or (scandir "/proc/self/fd"
+                              (lambda (file)
+                                (not (member file '("." "..")))))
+                     '()))))
+
+(define default-monitoring-period
+  ;; Default logging period, in seconds.
+  (make-parameter (* 20 60)))
+
+(define* (run-monitoring-service channel
+                                 #:key
+                                 (period (default-monitoring-period)))
+  (log-monitoring-stats)
+  (let loop ((period period))
+    (match (get-message* channel period 'log)
+      ('stop
+       (local-output (l10n "Terminating shepherd monitoring."))
+       #f)
+      ('log
+       (log-monitoring-stats)
+       (loop period))
+      (('set-period period)
+       (local-output (l10n "Monitoring logging period changed to ~a seconds.")
+                     period)
+       (loop period)))))
+
+(define* (spawn-monitoring-service #:key (period (default-monitoring-period)))
+  "Spawn the monitoring service and return a channel to communicate with it."
+  (let ((channel (make-channel)))
+    (spawn-fiber
+     (lambda ()
+       (run-monitoring-service channel
+                               #:period period)))
+    channel))
+
+(define* (monitoring-service #:key
+                             (period (default-monitoring-period)))
+  "Return a service that will monitor shepherd resource usage by printing it
+every @var{period} seconds."
+  (make <service>
+    #:docstring "Periodically log shepherd resource usage information."
+    #:provides '(monitoring)
+    #:requires '()
+    #:start (lambda args
+              (spawn-monitoring-service #:period period))
+    #:stop (lambda (channel)
+             (put-message channel 'stop)
+             #f)
+    #:actions
+    (make-actions
+     (period
+      "Set the logging period (in minutes) of the monitoring system."
+      (lambda (channel period)
+        (define (positive-integer? n)
+          (and (integer? n) (> n 0)))
+
+        (match (string->number period)
+          ((? positive-integer? period)
+           (put-message channel `(set-period ,period)))
+          (#f
+           (local-output
+            (l10n "~a: invalid number; expected a positive integer~%")
+            period)))))
+     (log
+      "Log monitoring info right away."
+      (lambda (channel)
+        (log-monitoring-stats))))))
diff --git a/po/POTFILES.in b/po/POTFILES.in
index 2e08939..8af7712 100644
--- a/po/POTFILES.in
+++ b/po/POTFILES.in
@@ -6,5 +6,6 @@ modules/shepherd/scripts/herd.scm
 modules/shepherd/scripts/reboot.scm
 modules/shepherd/support.scm
 modules/shepherd/service.scm
+modules/shepherd/service/monitoring.scm
 modules/shepherd/args.scm
 modules/shepherd.scm
diff --git a/tests/services/monitoring.sh b/tests/services/monitoring.sh
new file mode 100644
index 0000000..396f9f2
--- /dev/null
+++ b/tests/services/monitoring.sh
@@ -0,0 +1,62 @@
+# GNU Shepherd --- Test monitoring service.
+# Copyright © 2023 Ludovic Courtès <ludo@gnu.org>
+#
+# This file is part of the GNU Shepherd.
+#
+# The GNU Shepherd is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# The GNU Shepherd is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with the GNU Shepherd.  If not, see <http://www.gnu.org/licenses/>.
+
+shepherd --version
+herd --version
+
+socket="t-socket-$$"
+conf="t-conf-$$"
+log="t-log-$$"
+pid="t-pid-$$"
+
+herd="herd -s $socket"
+
+trap "cat $log || true;
+      rm -f $socket $conf $log;
+      test -f $pid && kill \`cat $pid\` || true; rm -f $pid" EXIT
+
+cat > "$conf" <<EOF
+(use-modules (shepherd service monitoring))
+
+(register-services (monitoring-service))
+EOF
+
+rm -f "$pid" "$log"
+shepherd -I -s "$socket" -c "$conf" -l "$log" --pid="$pid" &
+
+# Wait till it's ready.
+while ! test -f "$pid" ; do sleep 0.3 ; done
+
+$herd start monitoring
+$herd status monitoring | grep "started"
+
+n=0
+while ! grep "heap:" "$log" && test $n -lt 10
+do
+    sleep 1
+    n=$(expr $n + 1)
+done
+test $n -lt 10
+
+$herd log monitoring
+$herd log monitoring | grep "heap:"
+$herd period monitoring 1
+! $herd period monitoring not-a-number
+
+$herd stop monitoring
+$herd status monitoring | grep "stopped"