[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#39258] [PATCH v6 0/2] DRAFT "guix search" performances
From: |
Ludovic Courtès |
Subject: |
[bug#39258] [PATCH v6 0/2] DRAFT "guix search" performances |
Date: |
Fri, 23 Jul 2021 17:43:12 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hi!
zimoun <zimon.toutoune@gmail.com> skribis:
> This is an attempt to improve the performance of "guix search". It is still
> half baked but it allows to discuss further the idea about expanding the
> current '/lib/guix/package.cache' and avoids to forget an IRL discussion. ;-)
Thanks for resuming this discussion. :-)
> From my understanding, the issue that 'package-relevance' accepts a 'package'
> (and then all the chain until displaying) and 'fold-avaibale-packages' does
> not return a package. Well, I do not know; especially where to put something
> similiar to 'read-package-from'.
Yeah that’s annoying. Perhaps we need <proto-package> or
<package-metadata>. With some trickery we could have record type
inheritance or something, maybe. Dunno.
It would be good if we could arrange so that ‘fold-available-packages’
doesn’t allocate anything though.
> Let compare only for cold cache and time this cache building (Guix 7db8fd6):
>
> sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
> time guix build --check $(guix gc --derivers $(readlink -f
> ~/.config/guix/current/lib/guix/package.cache))
>
> real 0m28,848s
> user 0m1,481s
> sys 0m0,252s
>
> sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
> time guix build --check $(guix gc --derivers $(readlink -f
> /tmp/new/lib/guix/package.cache))
>
> real 0m40,279s
> user 0m1,582s
> sys 0m0,232s
>
> It seems longer but compared to the time of "guix pull" completion, it seems
> acceptable.
Both the initial timing and the target are waaay too much. :-/
On my i7 laptop I have:
--8<---------------cut here---------------start------------->8---
$ time ./pre-inst-env guile -c '(use-modules (gnu packages))
(generate-package-cache "/tmp/t.cache")'
real 0m20.738s
user 0m44.413s
sys 0m0.341s
--8<---------------cut here---------------end--------------->8---
It’s CPU-bound; we should probably start by optimizing that.
In Guile 3.0.7 there was a change that improved this noticeably:
https://git.savannah.gnu.org/cgit/guile.git/commit/?id=05614f792bfabbc33798863edd0bb67c744e9299
We should prolly look for similar optimization opportunities in the
assembler…
> Let compare for some queries:
>
> sudo sh -c 'echo 3 > /proc/sys/vm/drop_caches'
> time guix search game | recsel -C -P name | wc -l
> 371
>
> real 0m7,561s
> user 0m3,525s
> sys 0m0,391s
I think you should run:
time guix search game > /dev/null
otherwise Bash’s built-in ‘time’ shows the wall-clock time of the whole
pipeline, and the processing time of ‘recsel’ is probably not negligible
here.
[...]
> Last, two minor remarks about previous comments.
>
> 1. Ludo commented:
>
> > Therefore the cache '/lib/guix/package.cache' contains more
> > information.
>
> This breaks the binary interface, so we’ll have to analyze the impact
> of
> such a change and devise a strategy.
>
> <http://issues.guix.gnu.org/39258#93>
>
> and after some checking, this should be fine, IIUC. The '--news' is ok
> because of '#:allow-other-keys'. And other parts are modified accordingly.
> Guix revision N creates a cache that Guix revision N+1 will read but it should
> not be an issue; see 'inferior-available-packages'.
>
> 2. And Ludo wrote:
>
> I realize the other cache also has that problem, but it would be nice
> to add a
> version tag to the cache. Basically emit something like:
>
> (package-metadata-cache (version 0) VECTOR …)
>
> instead of just:
>
> (VECTOR …)
>
> <http://issues.guix.gnu.org/39258#93>
>
> which is, after discussions, not necessary. Versioning does not make sense
> here because the cache is read by the Guix which generates it. Therefore,
> specify a version is extraneous here.
I confirm! :-)
Thanks,
Ludo’.