[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] git-download: Speed up 'git-predicate'.
From: |
Christopher Baines |
Subject: |
Re: [PATCH] git-download: Speed up 'git-predicate'. |
Date: |
Fri, 2 Jun 2017 08:34:59 +0100 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 |
To provide some anecdotal evidence on the difference this makes, I wrote
a little test script [1]. It should roughly follow how git-predicate is
used with the local-file gexp.
Running the current implementation of git-predicate on the guix
repository with 50 trials takes:
real 0m5.374s
user 0m4.052s
sys 0m1.484s
Compared to the following for the patched implementation:
real 0m8.398s
user 0m5.040s
sys 0m2.016s
This shows that the patched implementation has decreased the
performance. Using the real value, the test time has increased by ~3
seconds. This corresponds roughly to a change from ~0.11 seconds
previously, to ~0.17 seconds for each individual traversal of the entire
repository.
I encountered a performance issue when trying to use git-predicate with
this repository [2], which I have written a Guix package for here [3].
Instead of the 1463 files that git ls-files reports locally for the guix
repository, smart-answers contains 26732 files.
The running time with smart-answers is left as an exercise to the
reader, all I can say is that on my machine, it takes more than 40
minutes for just 1 trial.
Using this patch, running the test script with 1 trial gives:
real 0m4.917s
user 0m3.640s
sys 0m1.428s
1:
(use-modules (srfi srfi-1)
(srfi srfi-26)
(ice-9 ftw)
(guix git-download))
(define (test file select?)
(let dump ((f file) (s (lstat file)))
(case (stat:type s)
((regular)
(display "."))
((directory)
(for-each
(lambda (e)
(let* ((f (string-append f "/" e))
(s (lstat f)))
(if (select? f s)
(dump f s))))
(scandir f (negate (cut member <> '("." ".."))) string<?)))
((symlink)
(display ".")))))
(define (run-test dir trials)
(chdir dir)
(for-each
(lambda (trial)
(test dir (git-predicate dir)))
(iota trials)))
(run-test (second (command-line))
(string->number (third (command-line))))
2: https://github.com/alphagov/smart-answers
3:
https://github.com/alphagov/govuk-guix/blob/65c6b8f3a0f01cd6ae4b51f356b74d4472b08e70/gds/packages/govuk.scm#L1136-L1153
signature.asc
Description: OpenPGP digital signature