01/02: offload: Modify the build-machine record to accept multiple systems.

[guix-commits]

apteryx pushed a commit to branch master
in repository guix.

commit 4b5a6fbc9b754c0ca70d033dd99f17c4f028733a
Author: Maxim Cournoyer <maxim.cournoyer@gmail.com>
AuthorDate: Mon Aug 24 16:26:14 2020 -0400

    offload: Modify the build-machine record to accept multiple systems.
    
    * guix/scripts/offload.scm (<build-machine>)[systems]: New field.
    [system]: Accessor changed to %build-machine-system.  Default to #f.
    * guix/scripts/offload.scm (build-machine-system): Wrap 
%build-machine-system
    with a deprecation warning.
    (build-machine-systems): Access the new systems field or fallback to use
    build-machine-system, for backward compatibility.
    (machine-matches?): Adjust.
    * tests/offload.scm: Add tests...
    * Makefile.am (SCM_TESTS): ...and register them.
    * doc/guix.texi (Daemon Offload Setup): Update doc.
---
 Makefile.am              |  1 +
 doc/guix.texi            | 35 +++++++++++++-----------
 guix/scripts/offload.scm | 24 +++++++++++++---
 tests/offload.scm        | 71 ++++++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 111 insertions(+), 20 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index 4e50a33f..9c38c2f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -433,6 +433,7 @@ SCM_TESTS =                                 \
   tests/monads.scm                             \
   tests/nar.scm                                \
   tests/networking.scm                         \
+  tests/offload.scm                            \
   tests/opam.scm                               \
   tests/openpgp.scm                            \
   tests/packages.scm                           \
diff --git a/doc/guix.texi b/doc/guix.texi
index 91d3860..0b79a49 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -1043,29 +1043,31 @@ When desired, the build daemon can @dfn{offload} 
derivation builds to
 other machines running Guix, using the @code{offload} @dfn{build
 hook}@footnote{This feature is available only when
 @uref{https://github.com/artyom-poptsov/guile-ssh, Guile-SSH} is
-present.}.  When that
-feature is enabled, a list of user-specified build machines is read from
-@file{/etc/guix/machines.scm}; every time a build is requested, for
-instance via @code{guix build}, the daemon attempts to offload it to one
-of the machines that satisfy the constraints of the derivation, in
-particular its system type---e.g., @file{x86_64-linux}.  Missing
-prerequisites for the build are copied over SSH to the target machine,
-which then proceeds with the build; upon success the output(s) of the
-build are copied back to the initial machine.
+present.}.  When that feature is enabled, a list of user-specified build
+machines is read from @file{/etc/guix/machines.scm}; every time a build
+is requested, for instance via @code{guix build}, the daemon attempts to
+offload it to one of the machines that satisfy the constraints of the
+derivation, in particular its system types---e.g., @code{x86_64-linux}.
+A single machine can have multiple system types, either because its
+architecture natively supports it, via emulation (@pxref{Transparent
+Emulation with QEMU}), or both.  Missing prerequisites for the build are
+copied over SSH to the target machine, which then proceeds with the
+build; upon success the output(s) of the build are copied back to the
+initial machine.
 
 The @file{/etc/guix/machines.scm} file typically looks like this:
 
 @lisp
 (list (build-machine
         (name "eightysix.example.org")
-        (system "x86_64-linux")
+        (systems (list "x86_64-linux" "i686-linux"))
         (host-key "ssh-ed25519 AAAAC3Nza@dots{}")
         (user "bob")
         (speed 2.))     ;incredibly fast!
 
       (build-machine
         (name "armeight.example.org")
-        (system "aarch64-linux")
+        (systems (list "aarch64-linux"))
         (host-key "ssh-rsa AAAAB3Nza@dots{}")
         (user "alice")
         (private-key
@@ -1075,8 +1077,8 @@ The @file{/etc/guix/machines.scm} file typically looks 
like this:
 
 @noindent
 In the example above we specify a list of two build machines, one for
-the @code{x86_64} architecture and one for the @code{aarch64}
-architecture.
+the @code{x86_64} and @code{i686} architectures and one for the
+@code{aarch64} architecture.
 
 In fact, this file is---not surprisingly!---a Scheme file that is
 evaluated when the @code{offload} hook is started.  Its return value
@@ -1096,8 +1098,9 @@ builds.  The important fields are:
 @item name
 The host name of the remote machine.
 
-@item system
-The system type of the remote machine---e.g., @code{"x86_64-linux"}.
+@item systems
+The system types the remote machine supports---e.g., @code{(list
+"x86_64-linux" "i686-linux")}.
 
 @item user
 The user account to use when connecting to the remote machine over SSH.
@@ -25025,7 +25028,7 @@ Maximum number of backup files to keep.
 Defaults to @samp{3}
 
 @end deftypevr
-
+@node Transparent Emulation with QEMU
 @subsubheading Transparent Emulation with QEMU
 
 @cindex emulation
diff --git a/guix/scripts/offload.scm b/guix/scripts/offload.scm
index 20ae7a9..a56701f 100644
--- a/guix/scripts/offload.scm
+++ b/guix/scripts/offload.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2014, 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès 
<ludo@gnu.org>
 ;;; Copyright © 2017 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -66,14 +67,16 @@
 ;;;
 ;;; Code:
 
-
 (define-record-type* <build-machine>
   build-machine make-build-machine
   build-machine?
   (name            build-machine-name)            ; string
   (port            build-machine-port             ; number
                    (default 22))
-  (system          build-machine-system)          ; string
+  (systems         %build-machine-systems         ; list of strings
+                   (default #f))                  ; drop default after system 
is removed
+  (system          %build-machine-system          ; deprecated
+                   (default #f))
   (user            build-machine-user)            ; string
   (private-key     build-machine-private-key      ; file name
                    (default (user-openssh-private-key)))
@@ -91,6 +94,19 @@
   (features        build-machine-features         ; list of strings
                    (default '())))
 
+;;; Deprecated.
+(define (build-machine-system machine)
+  (warning (G_ "The 'system' field is deprecated, \
+please use 'systems' instead.~%"))
+  (%build-machine-system machine))
+
+;;; TODO: Remove after the deprecated 'system' field is removed.
+(define (build-machine-systems machine)
+  (or (%build-machine-systems machine)
+      (list (build-machine-system machine))
+      (leave (G_ "The build-machine object lacks a value for its 'systems'
+field."))))
+
 (define-record-type* <build-requirements>
   build-requirements make-build-requirements
   build-requirements?
@@ -359,8 +375,8 @@ of free disk space on '~a'~%")
 
 (define (machine-matches? machine requirements)
   "Return #t if MACHINE matches REQUIREMENTS."
-  (and (string=? (build-requirements-system requirements)
-                 (build-machine-system machine))
+  (and (member (build-requirements-system requirements)
+               (build-machine-systems machine))
        (lset<= string=?
                (build-requirements-features requirements)
                (build-machine-features machine))))
diff --git a/tests/offload.scm b/tests/offload.scm
new file mode 100644
index 0000000..5a5de4e
--- /dev/null
+++ b/tests/offload.scm
@@ -0,0 +1,71 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (tests offload)
+  #:use-module (guix scripts offload)
+  #:use-module (srfi srfi-64))
+
+
+(test-begin "offload")
+
+(define-syntax-rule (expose-internal-definitions s1 s2 ...)
+  (begin
+    (define s1 (@@ (guix scripts offload) s1))
+    (define s2 (@@ (guix scripts offload) s2)) ...))
+
+(expose-internal-definitions machine-matches?
+                             build-requirements-system
+                             build-requirements-features
+                             build-machine-system
+                             build-machine-systems
+                             %build-machine-system
+                             %build-machine-systems
+                             build-machine-features)
+
+(define (deprecated-build-machine system)
+  (build-machine
+   (name "m1")
+   (user "dummy")
+   (host-key "some-key")
+   (system system)))
+
+(define (new-build-machine systems)
+  (build-machine
+   (name "m1")
+   (user "dummy")
+   (host-key "some-key")
+   (systems systems)))
+
+;;; Test that deprecated build-machine definitions still work.
+(test-assert (machine-matches? (deprecated-build-machine "i686-linux")
+                               (build-requirements
+                                (system "i686-linux"))))
+
+
+(test-assert (machine-matches? (new-build-machine '("i686-linux"))
+                               (build-requirements
+                                (system "i686-linux"))))
+
+;;; A build machine can act as more than one system type, thanks to QEMU
+;;; emulation.
+(test-assert (machine-matches? (new-build-machine '("armhf-linux"
+                                                    "aarch64-linux"
+                                                    "i686-linux"
+                                                    "x86_64-linux"))
+                               (build-requirements
+                                (system "armhf-linux"))))