[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Use of %texlive-revision and %texlive-tag in tex.scm
From: |
Bengt Richter |
Subject: |
Re: Use of %texlive-revision and %texlive-tag in tex.scm |
Date: |
Wed, 7 Jul 2021 10:38:44 +0200 |
User-agent: |
Mutt/1.10.1 (2018-07-13) |
On +2021-07-06 15:28:43 -0300, Nathan Benedetto Proença wrote:
> Bengt Richter <bokr@bokr.com> writes:
>
> > Hi Nathan,
> > Nice writeup!
>
> Thank you!
>
> > On +2021-07-05 11:03:46 -0300, Nathan Benedetto Proença wrote:
> >> Hello!
> >>
> >> I am trying to upgrade the package texlive, first for me, but hopefully
> >> for a patch, and I have a question regarding Guix policies.
> >>
> >> As you can see on
> >>
> >>
> >> https://git.savannah.gnu.org/cgit/guix.git/tree/guix/build-system/texlive.scm?id=f7692617b624a01615cf412773c4dad9a2deeb68
> >>
> >> the file guix/build-system/texlive.scm exposes two variables:
> >>
> >> (define %texlive-tag "texlive-2019.3")
> >> (define %texlive-revision 51265)
> >>
> >> These variables are used throughout gnu/packages/tex.scm, as you can see
> >> on
> >>
> >>
> >> https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/tex.scm?id=f7692617b624a01615cf412773c4dad9a2deeb68
> >>
> >> An example is the following code:
> >>
> >> (define hyph-utf8-scripts
> >> (origin
> >> (method svn-fetch)
> >> (uri (texlive-ref "generic" "hyph-utf8"))
> >> (file-name (string-append "hyph-utf8-scripts-"
> >> (number->string %texlive-revision)
> >> "-checkout"))
> >> (sha256
> >> (base32
> >> "0lk7shx768sxvgr85y8bnmmnj8x4bbkgpxrz3z8jp8avi33prw83"))))
> >>
> >> Grep tells me there are 290+ occurrences of `%texlive-revision`.
> >> What is the purpose of these variables?
> >>
> >> You see, they give me the impression that Guix is really concerned about
> >> upgrading *all* of texlive at once.
> >> These variables tell me I should go to the file texlive.scm and bump the
> >> tag and revision, and then handle all the broken hashes.
> >>
> >> Hence, it seems to me that any attempt to upgrade the texlive package
> >> would have to be done in a separate branch, which would only be merged
> >> into master when all the packages are upgraded.
> >>
> >> Is this the case?
> >> And if so, why?
> >>
> >> I have the impression that if such "monolithic" upgrade is not a goal,
> >> and "partial" our "per-package" upgrades are desirable, there may be
> >> better solutions.
> >>
> >> For example, we could add keyword arguments to texlive-ref and
> >> texlive-origin, so the code above becomes something like this
> >>
> >> (define hyph-utf8-scripts
> >> (origin
> >> (method svn-fetch)
> >> (uri (texlive-ref "generic" "hyph-utf8"
> >> #:texlive-tag "texlive-2019.3"
> >> #:texlive-revision 51265))
> >> (file-name "hyph-utf8-scripts-51625-checkout")
> >> (sha256
> >> (base32
> >> "0lk7shx768sxvgr85y8bnmmnj8x4bbkgpxrz3z8jp8avi33prw83"))))
> >>
> >> This would work right now, and we could eventually remove every use of
> >> %texlive-revision and %texlive-tag, so they become implementation
> >> details of the build-system texlive.scm; a fallback version.
> >> And further down the road we may even decide to remove this fallback,
> >> and make developers be explicit about their tags and revisions; this
> >> could amount to a refactor which makes the keyword arguments into
> >> required arguments, for example.
> >>
> >> I also like the second version of the code because the hash already
> >> pinpoints the tag and revision: both texlive-ref and texlive-origin use
> >> these variables to find the correct files to download.
> >> This just makes this dependency explicit.
> >>
> >> In any case, as this may be a choice between shipping stable and
> >> up-to-date packages, and as I am new to contributing to Guix, I found
> >> fitting to ask.
> >>
> >> Thanks in advance!
> >> Nathan
> >>
> >
> > I am wondering about guaranteeing generic behaviour by
> > guaranteeing program source and toolchain source hash
> > equivalences vs ignoring sources and guaranteeing end
> > results by testing results.
>
> It seems to me that you are talking about my email regarding using
> hashing in Scheme refactoring, right?
> This one:
>
> https://lists.gnu.org/archive/html/guix-devel/2021-07/msg00023.html
>
> I will assume this is the case, even though I we can actually simply
> talk about what you wrote.
>
> First thoughts: I think that what we really want to guarantee is
> "correctness", and that both hashes or tests are mere proxies for this.
>
> I see them as useful in distinct moments of package development.
>
> For example, lets say that I want to partially upgrade TeXLive (which I
> am already convinced is not a good idea, TBH, but serves as an
> example).
> To be able to do so, I may think that I want to refactor some function
> to pinpoint the version I will download.
> I can then make a single commit which updates the signature and all the
> call sites of the function.
> I can then actually upgrade the package I care about in a second commit,
> and do some "proper testing" in it, like producing some pdf with latex.
>
> My interest in checking hashes in the first commit is to get some
> confidence that this step, which may change hundred of places, did not
> introduce a new bug, so I can focus any debugging on the hopefully more
> local change made in the second commit.
>
> > I.e., if you want to print the sum of x and y passed as
> > strings to a program, output as a string to stdout, it
> > doesn't matter (other than optimization and debuggability)
> > what language the program was written in, so long as it was
> > compiled into a form that execve and co can launch and the
> > end result is the same.
> >
> > As part of testing, maybe strace could be used to generate
> > some kind of canonical kernel transaction trace that could
> > be used to compare behaviours for equivalency of executing
> > different-language programs?
> >
> > This would be a radical change in the approach to
> > reproducibility, maybe dynamically selecting from a
> > whitelist of trusted/tested substitutable executables with
> > hash names in /gnu but not necessarily (though not
> > excluding) binaries produced with guix source guarantees.
> >
> > Seems like guix is turing-complete enough to provide this
> > kind of substitutable foreign functions already, so might
> > this be a way to avoid mass recompilations?
> >
> > Or is this already available, but not so much used?
> >
> > I am not sure where to contibute thoughts like these, where
> > they would be of interest rather than distracting. (Pls
> > excuse the noise, if that's what this is to you).
>
> I am not sure I follow, although I do not regard this as noise.
>
> You are saying that one could use strace as a check that some program
> behaves the same as it did before, similar to the use that I suggested
> for hashing, right?
>
> I do not understand how this would be used, and how this avoids mass
> recompilations.
> I mean, to have binaries to inspect with strace I must build the code
> anyway, right?
> Am I missing something?
>
I had in mind expanding on and polishing something like
$ strace -s 72 ./bash-hello-strace |& grep ^write
write(1, "Hello strace!\n", 14Hello strace!
$ strace -s 72 ./guile-hello-strace.scm |& grep ^write
write(1, "Hello strace!\n", 14Hello strace!
Where
$ cat -n ./guile-hello-strace.scm |gxsnip
--8<---------------cut here---------------start------------->8---
1 #! /usr/bin/guile -s
2 !#
3 (display "Hello strace!\n")
--8<---------------cut here---------------end--------------->8---
and
$ cat -n ./bash-hello-strace |gxsnip
--8<---------------cut here---------------start------------->8---
1 #! /usr/bin/bash
2 echo 'Hello strace!'
--8<---------------cut here---------------end--------------->8---
I had in mind expanding on and polishing something like
$ strace -s 72 ./bash-hello-strace |& grep ^write
write(1, "Hello strace!\n", 14Hello strace!
$ strace -s 72 ./guile-hello-strace.scm |& grep ^write
write(1, "Hello strace!\n", 14Hello strace!
Where
$ cat -n ./guile-hello-strace.scm |gxsnip
--8<---------------cut here---------------start------------->8---
1 #! /usr/bin/guile -s
2 !#
3 (display "Hello strace!\n")
--8<---------------cut here---------------end--------------->8---
and
$ cat -n ./bash-hello-strace |gxsnip
--8<---------------cut here---------------start------------->8---
1 #! /usr/bin/bash
2 echo 'Hello strace!'
--8<---------------cut here---------------end--------------->8---
I had in mind expanding on and polishing something like
$ strace ./guile-hello-strace.scm |& grep ^write
write(1, "Hello strace!\n", 14Hello strace!
$ strace ./bash-hello-strace |& grep ^write
write(1, "Hello strace!\n", 14Hello strace!
Where
$ cat -n ./guile-hello-strace.scm |gxsnip
--8<---------------cut here---------------start------------->8---
1 #! /usr/bin/guile -s
2 !#
3 (display "Hello strace!\n")
--8<---------------cut here---------------end--------------->8---
and
$ cat -n ./bash-hello-strace |gxsnip
--8<---------------cut here---------------start------------->8---
1 #! /usr/bin/bash
2 echo 'Hello strace!'
--8<---------------cut here---------------end--------------->8---
Thus one could write something like this, compiling only it in whatever
environment,
so that updates to the tool chain which might otherwise have to be recompiled
can be ignored.
--
Regards
Bengt Richter