Re: substitute derivation: also substitute grafts?

[Maxime Devos]

On 15-09-2022 16:46, Csepp wrote:
> Ricardo Wurmus <rekado@elephly.net> writes:
>
> > [...]
> > Did I say *all items*? Well, … grafts are not included, because graft
> > derivations are marked as not substitutable.
> >
> > Can we change that conditionally? I would really like to avoid having
> > to build grafts on B when they have already been built on A.
>
> I would love this too, because IO can be incredibly slow on HDDs and
> large packages.  My netbook would be thankful.

There are some some opportunities for optimizations in the grafting code 
before substituting more -- for example, to avoid seek times, it would 
be possible to rewrite multiple files concurrently (maybe using 
'par-for-each' to process each file in a directory in parallel).

This is already done in (guix build grafts), except that:

 * 'find-files' itself is not parallelised, even though parallelising it
    could potentially reduce the time spent seeking (*)
 * it uses (parallel-job-count), which is (IIUC) 1 by default because
   --cores=1 by default.  As grafts are more IO intensive than CPU
   intensive, maybe it would be reasonable to impose a _minimum_ amount
   of parallelism, I don't, know, 2 or 4 or so?

(I'm assuming the main problem here is seek times.)

Also combinable with the proposal of having substitutes for grafts.

Greetings,
Maxime.

(*) IIUC, the time for N parallel seeks should, in theory, ≃ 1 seek, for 
small values of N, because of elevator algorithms.

OpenPGP_0x49E3EE22191725EE.asc
Description: OpenPGP public key

OpenPGP_signature
Description: OpenPGP digital signature