[bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile

[zimoun]

Hi Ludo,

On Wed, 27 Jan 2021 at 14:14, Ludovic Courtès <ludo@gnu.org> wrote:
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Here’s a simple but oft-requested feature (I remember discussing
>> with Pierre and Simon a year ago at the Guix Days about ways to
>> implement it—time flies!): these patches provide ‘guix package
>> --export-manifest’ and ‘--export-channels’.  These options spit
>> a manifest and a channel spec, respectively, with the goal of
>> helping users who wish to migrate to the declarative model.
>
> I’m rather happy with this patch set but since this is something we’ve
> discussed several times in the past, I think it’d be great if those
> interested could chime in and comment:
>
>   https://issues.guix.gnu.org/45919
>
> I’ll leave a few more days and then… push!

You are looking at me? :-)

Well, the feature is nice!  At first, I thought that it could be a bit
smarter than using only one commit.  But as you said, the aim is for
transitioning.  Maybe a future improvement should to list somewhere in
the comments which commit provides which set of packages.  It could
help… or not. :-)

Using Docker, it works.  Let describe how in case people are interested.


On machine A, I have:

--8<---------------cut here---------------start------------->8---
$ guix describe -f channels
(list (channel
        (name 'guix)
        (url "https://git.savannah.gnu.org/git/guix.git";)
        (commit
          "cb68ae668af2ade4b0777d82f227e5462768e9e5")
        (introduction
          (make-channel-introduction
            "9edb3f66fd807b096b48283debdcddccfea34bad"
            (openpgp-fingerprint
              "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA")))))

$ cat ~/.config/guix/manifests/python.scm
(specifications->manifest
 (append
  '("python"
    )
  (map
   (lambda (pkg)
     (string-append "python-" pkg))
   '("ipython"
     "numpy"
     "matplotlib"
     "scipy"
     "biopython"
  ))))
--8<---------------cut here---------------end--------------->8---

And I generate a Docker pack with:

--8<---------------cut here---------------start------------->8---
$ guix pack -f docker --save-provenance -m ~/.config/guix/manifests/python.scm
/gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz
--8<---------------cut here---------------end--------------->8---


Then on machine B, after fetching this tarball, I run:

--8<---------------cut here---------------start------------->8---
$ docker load < /tmp/img/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz
$ docker images
REPOSITORY                           TAG                 IMAGE ID            
CREATED             SIZE
python-python-ipython-python-numpy   latest              49ddfedf1e27        51 
years ago        1.45GB
--8<---------------cut here---------------end--------------->8---

And it works as expected:

--8<---------------cut here---------------start------------->8---
$ docker run -ti python-python-ipython-python-numpy:latest python3
Python 3.8.2 (default, Jan  1 1970, 00:00:01)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>> import numpy as np
>>> A = np.array([[1,0,1],[0,1,0],[0,0,1]])
>>> _, s, _ = np.linalg.svd(A); s; abs(s[0] - 1./s[2])
array([1.61803399, 1.        , 0.61803399])
0.0
>>>
--8<---------------cut here---------------end--------------->8---

Neat!

So far, so good.  Well, let extract the ’manifest’ from this Docker
blob.

--8<---------------cut here---------------start------------->8---
$ docker export -o /tmp/img/re-pack.tar $(docker ps -a --format "{{.ID}}" | 
head -n1)
$ tar -xf /tmp/img/re-pack.tar $(tar -tf /tmp/img/re-pack.tar | grep 
'profile/manifest')
$ cat gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile/manifest | grep -E 
"(\(\"python|cb68ae)" | head -n5
    (("python"
              "cb68ae668af2ade4b0777d82f227e5462768e9e5")
     ("python-ipython"
        (("python-backcall"
         ("python-pyzmq"
--8<---------------cut here---------------end--------------->8---

Now, a trick to get the channels and specifications:

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix package -p 
/tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-channels
;; This channel file can be passed to 'guix pull -C' or to
;; 'guix time-machine -C' to obtain the Guix revision that was
;; used to populate this profile.

(list
     (channel
       (name 'guix)
       (url "https://git.savannah.gnu.org/git/guix.git";)
       (commit
         "cb68ae668af2ade4b0777d82f227e5462768e9e5")
       (introduction
         (make-channel-introduction
           "9edb3f66fd807b096b48283debdcddccfea34bad"
           (openpgp-fingerprint
             "BBB0 2DDF 2CEA F6A8 0D1D  E643 A2A0 6DF2 A33A 54FA"))))
)

$ ./pre-inst-env guix package -p 
/tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-manifest
$ ./pre-inst-env guix package -p 
/tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-manifest
;; This "manifest" file can be passed to 'guix package -m' to reproduce
;; the content of your profile.  This is "symbolic": it only specifies
;; package names.  To reproduce the exact same profile, you also need to
;; capture the channels being used, as returned by "guix describe".
;; See the "Replicating Guix" section in the manual.

(specifications->manifest
  (list "python"
        "python-ipython"
        "python-numpy"
        "python-matplotlib"
        "python-scipy"
        "python-biopython"))
--8<---------------cut here---------------end--------------->8---

Awesome!


The unexpected is this channels and manifests files do not reproduce the
same Docker pack tarball:

--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 99   Jan 05 2021 16:56:39    (current)
  guix-past 829923f
    repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
    branch: master
    commit: 829923f01f894f1e687735627025ada26230832f
  guix-bimsb a8b539d
    repository URL: https://github.com/BIMSBbioinfo/guix-bimsb
    branch: master
    commit: a8b539d61a359060c35f3cb34c7edd1d9d14241d
  bimsb-nonfree 4084e63
    repository URL: https://github.com/BIMSBbioinfo/guix-bimsb-nonfree.git
    branch: master
    commit: 4084e63c9c0d662780870aded9f5a6ca1b063780
  guix-science cf87b05
    repository URL: https://github.com/guix-science/guix-science.git
    branch: master
    commit: cf87b0501c4a38b96edf41025a27bf1cb91f521a
  guix 957f0c4
    repository URL: https://git.savannah.gnu.org/git/guix.git
    branch: master
    commit: 957f0c40327ce00f53db22737e3775ce616ac258

$ guix time-machine -C /tmp/img/channels.scm -- pack -f docker 
--save-provenance -m /tmp/img/manifest.scm
Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...
/gnu/store/xzk604g8gysv4azn7sf9nylr6iah97gl-docker-pack.tar.gz
--8<---------------cut here---------------end--------------->8---

To compare with
/gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz.

On a third machine, I get:
/gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz

Well, that’s another story and I have not inspected yet the
derivations and what could be wrong on the machine B.


Cheers,
simon