bug#42009: package.cache not deterministic

[zimoun]

Hi Ludo,

On Mon, 29 Jun 2020 at 14:34, Ludovic Courtès <ludo@gnu.org> wrote:

> I realize I was a bit off-topic: I was commenting on the more general
> issue of .go non-reproducibility.
>
> The problem with the package cache seems to be different.  Sorry for the
> confusion!

Well, the .go non-reproducibility is nonetheless interesting. :-)

> So, surprisingly, it’s not just an ordering issue: the caches do contain
> different pieces of information.

Interestingly, on my machine with 8ea91d0, I have 12 differences but
none of them is Python.  Instead, I have: "rust", "ocaml",
"mingw-w64-i686", "clang-toolchain", "clang-runtime", "linux-libre",
"icedtea", "gcc-objc++", "gcc-objc", "bdb", "rust-lazy-static",
"gcc-toolchain".  All are aliases and python-2 is too but does not
appear...
On the other hand, these 2 aliases do not appear either in your list.

--8<---------------cut here---------------start------------->8---
(define-public clang clang-9)
(define-public clang-toolchain clang-toolchain-9)
--8<---------------cut here---------------end--------------->8---

The question is: do we need to declare public e.g. clang-9?  Declare
clang is it not enough already?

For example, ocaml-4.09 is not used outside gnu/packages/ocalm.scm.
Idem for ghc-8.6, etc..

> This patch solves the ordering issue

The 'sort' will not help for improving the speed of "guix pull". ;-)

> But it’s not quite right because the order in which variables are
> traversed is still non-deterministic, so between two runs of
> ‘generate-package-cache’, caches differ like this:

It depends on the eddy current in the upper atmosphere. :-)
https://xkcd.com/378/

Well, it finds one or the other first and 'expand-cache' stores the
first considering the second as "seen", isn't it?

> The problem has to do with aliases: we don’t always see the same
> variable first.  So we have to sort before calling ‘expand-cache’.

The question is why it is not always the same variable first.  Well,
IMHO, it could comes from:
 - either 'scandir*' in 'scheme-files' because all the other functions
seem "pure" and this one not;
 - either 'fold-module-public-variables*' where 'seen' is one or the other.
Or if the .go files are not deterministic, especially about 'gensym',
it should explain why one symbol appears sometimes first.

Cheers,
simon