bug#39885: Bioconductor URI, fallback and time-machine

[zimoun]

Hi,

On Tue, 03 Mar 2020 at 16:59, zimoun <zimon.toutoune@gmail.com> wrote:

> Currently, the URI scheme (see 'bioconductor-uri' in
> guix/build-system/r.scm) is:
>
>  
> https://bioconductor.org/packages/release/data/<type-url-part>/src/contrib/<upstream-name>-<version>.tar.gz
>
> which leads to 2 issues:
>
>  1. when Bioconductor updates their release, some package versions are
> updated too, and so, the upstream return 404.
>  2. for this reason 1., the "guix time-machine" is broken for all the
> Bioconductor packages, at least if Berlin or SWH does not have a
> substitute; which is not expected for 'annotation' packages.
>
> However, the Bioconductor archive still serves the old release, i.e.,
>
> https://bioconductor.org/packages/3.x/data/<type-url-part>/src/contrib/<upstream-name>-<version>.tar.gz

It is still the case and for concrete breakage, see [1].  I will not
detail but each time Guix lags behind Bioconductor new release, it is
broken.  For sure, Guix upgrades more or less quickly.  Each time
Bioconductor remove a package, it is broken.  Well, because a lot
of care about R packages, the forward breakages happen barely. :-)  But
backward breakages are not negligible, IMHO.


Well, this URL choice is not The Right Thing and somehow broken by design.

1: <https://issues.guix.gnu.org/39885#7>


> The ways to fix the both issues are:
>
>  a) Add the Bioconductor release (known at packaging time) to all the
> packages; provide as argument to 'bioconductor-uri'.
>  b) Add more URLs to fallback.
>
> As discussed on IRC, Tobias seems more inclined with the option a) and
> I am more in favour of option b.
>
> Attached, a quick patch showing the option b).

We are now 1.5 years after.  And we did nothing; well we did other
things instead. ;-).  Now, I have an strong opinion that option a) is
not doable: I speak using my janitor moves of Bioconductor packages.

Instead, something along the proposed patch below half-fixes the issue
now.  We just have to append the releases and let the fallback mechanism
takes care.  It reduces the maintenance burden, IMHO.

For sure, it is not perfect but it appears to me a pragmatical fix
waiting something better.


This better is unknown (at least from me :-)).  On one hand Disarchive
would improve the situation for tarballs… but some work remains (check
that SWH ingestion and rebuild is bullet-proof).  On the other hand,
Bioconductor uses Git, for instance:

    git clone https://git.bioconductor.org/packages/CATALYST

<https://bioconductor.org/packages/release/bioc/html/CATALYST.html>

And Bioconductor uses ’origin/RELEASE_3.14’ as Git tag.  Based on this,
it would avoid the eternal inplace-change fixes.

For instance, the package tximeta [2], recently updated by Ricardo.
Well, from their Bioconductor Git repo,

    git clone https://git.bioconductor.org/packages/tximeta

it is not clear that the current version is at 1.12.3.  And it is not
clear either if they tagged origin/RELEASE_3_14 at 1.12.0 and did
something ugly to then get 1.12.3.  Anyway, switch from url-fetch to
git-fetch is an option.  However, it is as option a) and I am not
convinced it is doable with the resource at hand.

2: <https://bioconductor.org/packages/3.14/bioc/html/tximeta.html>


What could a plan to have a bullet-proof “guix time-machine” for
Bioconductor?


Cheers,
simon


> From 87e73e02202fe5e342d68f1fb17efdd4425737cd Mon Sep 17 00:00:00 2001
> From: zimoun <zimon.toutoune@gmail.com>
> Date: Tue, 3 Mar 2020 16:53:39 +0100
> Subject: [PATCH] build-system: r: Use Bioconductor old releases to fallback.
>
> * guix/build-system/r.scm (bioconductor-uri): Extend the fallback list.
> ---
>  guix/build-system/r.scm | 21 ++++++++++++---------
>  1 file changed, 12 insertions(+), 9 deletions(-)
>
> diff --git a/guix/build-system/r.scm b/guix/build-system/r.scm
> index 2d328764b0..8638e1b888 100644
> --- a/guix/build-system/r.scm
> +++ b/guix/build-system/r.scm
> @@ -54,15 +54,18 @@ release corresponding to NAME and VERSION."
>                           ('annotation "/data/annotation")
>                           ('experiment "/data/experiment")
>                           (_ "/bioc"))))
> -    (list (string-append "https://bioconductor.org/packages/release";
> -                         type-url-part
> -                         "/src/contrib/"
> -                         name "_" version ".tar.gz")
> -          ;; TODO: use %bioconductor-version from (guix import cran)
> -          (string-append "https://bioconductor.org/packages/3.10";
> -                         type-url-part
> -                         "/src/contrib/Archive/"
> -                         name "_" version ".tar.gz"))))
> +    (append (list (string-append "https://bioconductor.org/packages/release";
> +                                 type-url-part
> +                                 "/src/contrib/"
> +                                 name "_" version ".tar.gz"))
> +            (map (lambda (release)
> +                   (string-append "https://bioconductor.org/packages/";
> +                                  release
> +                                  type-url-part
> +                                  "/src/contrib/"
> +                                  name "_" version ".tar.gz"))
> +                 (list (@@ (guix import cran) %bioconductor-version)
> +                       "3.9" "3.8" "3.7")))))
>
>  (define %r-build-system-modules
>    ;; Build-side modules imported by default.