bug#39885: Bioconductor URI, fallback and time-machine

[zimoun]

Hi,

Some explanations of the issue are provided here:

    <http://issues.guix.gnu.org/issue/39885>

Since we are currently updating to 3.12, maybe it is the occasion to fix
the issue.  See option a) below.


On Tue, 03 Mar 2020 at 16:59, zimoun <zimon.toutoune@gmail.com> wrote:

> Currently, the URI scheme (see 'bioconductor-uri' in
> guix/build-system/r.scm) is:
>
>  
> https://bioconductor.org/packages/release/data/<type-url-part>/src/contrib/<upstream-name>-<version>.tar.gz
>
> which leads to 2 issues:
>
>  1. when Bioconductor updates their release, some package versions are
> updated too, and so, the upstream return 404.
>
>  2. for this reason 1., the "guix time-machine" is broken for all the
> Bioconductor packages, at least if Berlin or SWH does not have a
> substitute; which is not expected for 'annotation' packages.

An example of this issue is for example:

--8<---------------cut here---------------start------------->8---
$ guix time-machine --commit=aee183e -- import cran -a bioconductor CATALYST -r

Updating channel 'guix' from Git repository at 
'https://git.savannah.gnu.org/git/guix.git'...

Starting download of /tmp/guix-file.Nxajqh
>From 
>https://bioconductor.org/packages/release/bioc/src/contrib/CATALYST_1.12.2.tar.gz...
download failed 
"https://bioconductor.org/packages/release/bioc/src/contrib/CATALYST_1.12.2.tar.gz";
 404 "Not Found"
failed to download "/tmp/guix-file.Nxajqh" from 
"https://bioconductor.org/packages/release/bioc/src/contrib/CATALYST_1.12.2.tar.gz";
error: failed to retrieve package information from 
"https://cran.r-project.org/web/packages/CATALYST/DESCRIPTION": 404 ("Not 
Found")
Backtrace:
           4 (primitive-load "/home/simon/.cache/guix/inferiors/vznc…")
In guix/ui.scm:
  2117:12  3 (run-guix-command _ . _)
In guix/scripts/import.scm:
   120:11  2 (guix-import . _)
In srfi/srfi-1.scm:
   586:17  1 (map1 (#f))
In guix/import/utils.scm:
    258:2  0 (package->definition _)

guix/import/utils.scm:258:2: In procedure package->definition:
Throw to key `match-error' with args `("match" "no matching pattern" #f)'.
--8<---------------cut here---------------end--------------->8---

Aside the ugly backtrace which is tracked by #44115, the main issue is
because Bioconductor updated to 3.12 and Guix is still at 3.11.

Concretely, the issue is that ’release’ in the URL:

<https://bioconductor.org/packages/release/bioc/src/contrib/CATALYST_1.12.2.tar.gz>

now refers to 3.12 (because Bioconductor update) and Guix still think it
is 3.11 (because Guix has not yet updated; work-in-progress).  And
CATALYST in 3.12 is at version 1.14.0 against 1.12.2 for 3.11.
Therefore, the conflict and the error.

It means that while:

    (define %bioconductor-version "3.11")

is not updated to 3.12, all the Bioconductor packages are broken; in the
meaning not buildable from source.


>  a) Add the Bioconductor release (known at packaging time) to all the
> packages; provide as argument to 'bioconductor-uri'.
>  b) Add more URLs to fallback, e.g.:
>
> https://bioconductor.org/packages/release/data/<type-url-part>/src/contrib/<upstream-name>-<version>.tar.gz
> https://bioconductor.org/packages/3.11/data/<type-url-part>/src/contrib/<upstream-name>-<version>.tar.gz
>
> Attached, a quick patch showing the option b).

Then each time we update Bioconductor, we add an URL to the list.


> As discussed on IRC, Tobias seems more inclined with the option a) and
> I am more in favour of option b.

Tobias and Ricardo are in favor for a) (see this thread).  Which means a
lot of work IMHO, i.e., add 3.11 as arguments and then 3.12 to all the
Bioconductor packages and fix the importer, IIUC; while b) means do
nothing except merge the proposed patch (possibly re-worked).

Just to note that only the task to group in bioconductor.scm all the
Bioconductor packages scattered here and there is still not done, I
think option a) is not doable by hand – I do not volunteer! :-) Else,
any suggestion to script the task instead?

Since I am more in favor of b), I am less motivated to fix the a). ;-)
But I am motivated to fix the issue at hand. :-)


Other option c) is to switch all the Bioconductor to git-fetch instead
of url-fetch.  I have not checked yet how could be the transition.


> Please also consider #36805 which was never merged or closed.
>  http://issues.guix.gnu.org/issue/36805

This patch could help for option a).


WDYT?

All the best,
simon