Re: Build dependency inflation (was: Re: Core-updates merge)

[Christopher Baines]

Simon Josefsson via "Development of GNU Guix and the GNU System distribution." 
<guix-devel@gnu.org> writes:

> [[PGP Signed Part:Undecided]]
> Andreas Enge <andreas@enge.fr> writes:
>
>> - Too much in Guix depends on too much else, which makes building things
>>   needlessly entangled; in particular time zone data should not be referred
>>   to by packages, but be loaded at runtime (Leo Famulari).
>
> This is an important open problem -- is there any way to attack this
> problem in a systematic way?  I guess it is hard to understand which
> packages ends up depending on what since it is a large graph with long
> cycles, and also to understand which build depencies are essential and
> which are superficial, and thus consequently challenging to know where
> to start working to reduce build dependencies?
>
> I wonder if it is possible to graph all the build dependencies, and do
> something like a monte-carlo what-if simulation: randomly pick one
> build-dependency from the entire build-dependency graph and remove it,
> and recompute the graph.  If the difference between these two graphs
> leads to a significantly lower total build computational cost than
> before, we may be on to something.  My theory is that "true" build
> dependencies will show up in so many places that removing just one
> instance of it will not affect total build time.  But "needless" build
> dependencies may occur just once or few times, and this approach would
> catch low-hanging fruit to work on.  Maybe the simulation needs to be
> done on more than just removing one build-dependency, it could play
> what-if games removing two build-dependencies etc, or three random
> build-dependencies, and so on.  Maybe my idea is flawed, and this will
> only lead to a list of build-dependencies that are impossible to get rid
> off anyway.
>
> Is there some other method to understand what build dependencies would
> be important to remove, to speed up total rebuild time?
>
> Maybe we could analyze how much of a particular build-dependency
> actually is used by a particular build?  By looking into file-access
> patterns etc.

This is something I'd like to see incorporated in to qa.guix.gnu.org for
patch (and branch) review. While one off analysis is good, I think it's
most important to be able to look at changes and see how they change the
situation.

signature.asc
Description: PGP signature