[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#34060] [PATCH 00/10] Add a cache for package lookups
From: |
Ludovic Courtès |
Subject: |
[bug#34060] [PATCH 00/10] Add a cache for package lookups |
Date: |
Sun, 13 Jan 2019 16:45:32 +0100 |
Hello!
This patch adjust ‘guix pull’ & co. such that Guix comes with a
cache to speed up package lookups. The end result is that:
guix build coreutils
becomes equivalent to:
guix build -e '(@ (gnu packages base) coreutils)'
That means we have fewer files to open (only the closure of
(gnu packages base) instead of all the (gnu packages …) modules),
and thus less I/O and a smaller memory footprint. Without cache,
we have:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./pre-inst-env guix build coreutils -nd
/gnu/store/ddlgwpishzb8985gwg5fdrydvllg254a-coreutils-8.30.drv
Garbage collection statistics:
heap size: 46.18 MiB
allocated: 102.57 MiB
GC times: 14
time spent in GC: 0.25 seconds (25% of user time)
1.01user 0.06system 0:00.96elapsed 112%CPU (0avgtext+0avgdata
158916maxresident)k
0inputs+0outputs (0major+14602minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
With the cache, we get:
--8<---------------cut here---------------start------------->8---
$ GUIX_PROFILING=gc time ./foo/bin/guix build coreutils -nd
/gnu/store/ddlgwpishzb8985gwg5fdrydvllg254a-coreutils-8.30.drv
Garbage collection statistics:
heap size: 36.52 MiB
allocated: 87.30 MiB
GC times: 14
time spent in GC: 0.24 seconds (27% of user time)
0.89user 0.06system 0:00.84elapsed 112%CPU (0avgtext+0avgdata
124168maxresident)k
0inputs+0outputs (0major+11423minor)pagefaults 0swaps
--8<---------------cut here---------------end--------------->8---
As a bonus, we take advantage of the cache to speed up ‘guix
package -A’ (which now runs in ~0.5s instead of ~1.7s) and
‘guix edit’.
The cache is populated by packages of all the channels specified
during ‘guix pull’.
The cache is *not* used at all when GUIX_PACKAGE_PATH is set or
a ‘-L’ flag is passed, or ./pre-inst-env is used.
Feedback welcome!
Ludo’.
Ludovic Courtès (10):
profiling: Add a "gc" profiling component.
guix package: Avoid 'find-newest-available-packages'.
packages: Remove 'find-newest-available-packages'.
inferior: Add 'gexp->derivation-in-inferior'.
discovery: Add 'fold-module-public-variables*'.
pull: Build profile with 'channel-instances->derivation'.
channels: Compute a package cache and use it.
edit: Use 'specification->location' to read information from the
cache.
guix package: '--list-available' can use data from the cache.
status: Distinguish 'package-cache' profile hook.
gnu/packages.scm | 253 ++++++++++++++++++++++++++++++++++-----
guix/channels.scm | 32 ++++-
guix/discovery.scm | 28 ++++-
guix/inferior.scm | 26 ++++
guix/profiling.scm | 25 +++-
guix/scripts/edit.scm | 29 ++---
guix/scripts/package.scm | 137 +++++++++++----------
guix/scripts/pull.scm | 4 +-
guix/status.scm | 2 +
tests/packages.scm | 77 ++++++++++--
10 files changed, 491 insertions(+), 122 deletions(-)
--
2.20.1
- [bug#34060] [PATCH 00/10] Add a cache for package lookups,
Ludovic Courtès <=
- [bug#34060] [PATCH 01/10] profiling: Add a "gc" profiling component., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 03/10] packages: Remove 'find-newest-available-packages'., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 04/10] inferior: Add 'gexp->derivation-in-inferior'., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 02/10] guix package: Avoid 'find-newest-available-packages'., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 10/10] status: Distinguish 'package-cache' profile hook., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 05/10] discovery: Add 'fold-module-public-variables*'., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 06/10] pull: Build profile with 'channel-instances->derivation'., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 08/10] edit: Use 'specification->location' to read information from the cache., Ludovic Courtès, 2019/01/13
- [bug#34060] [PATCH 09/10] guix package: '--list-available' can use data from the cache., Ludovic Courtès, 2019/01/13