[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#41702: `guix environment` performance issues
From: |
Ludovic Courtès |
Subject: |
bug#41702: `guix environment` performance issues |
Date: |
Sat, 27 Jun 2020 23:20:27 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) |
Hi,
Lars-Dominik Braun <ldb@leibniz-psychology.org> skribis:
> That would have been my best guess too, but it does not seem to be the biggest
> problem right now. Looking at the numbers again (both patches applied) with
> the
> attached manifest[1], I see that:
>
> ---snip---
> Local UNIX socket with and without --no-grafts:
> N Min Max Median Avg Stddev
> x 10 6.07 6.35 6.145 6.16 0.08232726
> + 10 17.47 17.89 17.545 17.602 0.14351152
> Difference at 99.0% confidence
> 11.442 +/- 0.150576
> 185.747% +/- 4.07133%
>
> Local UNIX socket vs. guix://localhost transport:
> N Min Max Median Avg Stddev
> x 10 17.47 17.89 17.545 17.602 0.14351152
> + 10 17.43 18.1 17.61 17.642 0.20131788
> No difference proven at 99.0% confidence
>
> Local UNIX socket vs ssh://localhost transport:
> N Min Max Median Avg Stddev
> x 10 17.47 17.89 17.545 17.602 0.14351152
> + 10 33.46 35.27 34.315 34.359 0.53873205
> Difference at 99.0% confidence
> 16.757 +/- 0.5074
> 95.1994% +/- 3.13957%
> ---snap---
>
> So I would conclude:
>
> 1) Grafting still takes a lot of time and needs more work
Yes, I noticed that there’s still redundant work being done, so we may
need global caching in addition to what
58bb833365db4e8934a386497d5b00a063cfd27d did.
> 2) Linux optimizes localhost networking pretty well
> 3) Our SSH transport is terribly slow
Yes.
The patch below is a noticeable improvement for me. On my laptop,
GUIX_DAEMON_SOCKET=ssh://localhost ./pre-inst-env guix build libreoffice -n
goes from 5.8s to 3.3s. It just does the same thing as we do for
guix://.
Could you check what results it gives you?
> So, I guess it would make sense for me to look at the SSH transport itself
> again and see if there are any other low-hanging fruit. Not sure how much I
> can
> help with profiling guile/guix itself. A different/better RPC protocol is
> probably GSoC/v2.0-worthy?
There’s a project to rewrite the daemon in Scheme, started by Caleb
Ristvedt (reepa) as part of GSoC a few years ago. This could be an
opportunity to add a new version of the protocol that would support
pipelining.
Thanks,
Ludo’.
diff --git a/guix/ssh.scm b/guix/ssh.scm
index 2d7ca7d01d..a9312b7c8c 100644
--- a/guix/ssh.scm
+++ b/guix/ssh.scm
@@ -98,7 +98,7 @@ actual key does not match."
key type))))))))
(define* (open-ssh-session host #:key user port identity
- host-key
+ host-key open-connection
(compression %compression)
(timeout 3600))
"Open an SSH session for HOST and return it. IDENTITY specifies the file
@@ -110,6 +110,10 @@ When HOST-KEY is true, it must be a string like
\"ssh-ed25519 AAAAC3Nz…
root@example.org\"; the server is authenticated and an error is raised if its
host key is different from HOST-KEY.
+When OPEN-CONNECTION is true, it must be a two-argument procedure; it is
+passed HOST and PORT and must return a socket (a file port). When
+OPEN-CONNECTION is false, Guile-SSH takes care of opening the connection.
+
Install TIMEOUT as the maximum time in seconds after which a read or write
operation on a channel of the returned session is considered as failing.
@@ -134,6 +138,13 @@ Throw an error on failure."
;; Honor ~/.ssh/config.
(session-parse-config! session)
+ (when open-connection
+ (let* ((sock (open-connection host port)))
+ ;; Since 'session-set!' doesn't increase the revealed count of SOCK
+ ;; (as of Guile-SSH 0.12.0), do it ourselves.
+ (set-port-revealed! sock 1)
+ (session-set! session 'fd sock)))
+
(match (connect! session)
('ok
(if host-key
diff --git a/guix/store/ssh.scm b/guix/store/ssh.scm
index 09c0832505..d07f0d7467 100644
--- a/guix/store/ssh.scm
+++ b/guix/store/ssh.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2017 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2017, 2020 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -19,6 +19,7 @@
(define-module (guix store ssh)
#:use-module (guix ssh)
#:use-module (web uri)
+ #:use-module (ssh session)
#:export (connect-to-daemon))
;;; Commentary:
@@ -29,11 +30,27 @@
;;;
;;; End:
+(define (open-connection host port)
+ "Open a connection to HOST and PORT. Use the standard SSH port if PORT is
+false."
+ (let* ((lst (getaddrinfo host
+ (if port (number->string port) "ssh")
+ (if port AI_NUMERICSERV 0)))
+ (addr (addrinfo:addr (car lst)))
+ (sock (socket (sockaddr:fam addr) SOCK_STREAM 0)))
+ ;; Setting this option makes a dramatic difference because it avoids the
+ ;; "ACK delay" on our RPC messages.
+ (setsockopt sock IPPROTO_TCP TCP_NODELAY 1)
+
+ (connect sock addr)
+ sock))
+
(define (connect-to-daemon uri)
"Connect to the SSH daemon at URI, a URI object with the 'ssh' scheme."
(remote-daemon-channel
(open-ssh-session (uri-host uri)
#:port (or (uri-port uri) 22)
- #:user (uri-userinfo uri))))
+ #:user (uri-userinfo uri)
+ #:open-connection open-connection)))
;;; ssh.scm ends here