G expressions

[Ludovic Courtès]

Hello!

1. The Problem
===============

There’s a recurrent problem that we need to communicate the file name of
store items to Scheme code that is going to live in another process:
expressions to build a derivation, Scheme files that are to be loaded by
other processes, etc.

This had been partly addressed by having ‘build-expression->derivation’
install two global variables in the build code: ‘%build-inputs’ and
‘%outputs’.

However, for generated Scheme files (as in (gnu system) and related
modules), there’s no such mechanism.  Thus, either we use something
like:

  (mlet %store-monad ((ls (package-file coreutils "bin/ls")))
    (text-file "foo" (object->string `(system* ,ls))))

but then the problem is that the resulting text file doesn’t hold a
reference to Coreutils, which is wrong.  Or, we do something like:

  (text-file* "foo" "(system* \"" coreutils "/bin/ls\")")

The resulting file does have a reference to Coreutils, but the approach
obviously sucks.

Besides, ‘%build-inputs’ is not particularly elegant either, and it’s
error-prone (if you refer to an input by the wrong name, you only notice
when building, with an obscure wrong-type-arg error.)  That’s been OK as
it’s only used occasionally in package recipes.


2. G Expressions
=================

“G-expressions”, or gexps, are meant to address these two issues
mentioned above:

  1. Replacing a reference to a package or derivation by its output file
     name;

  2. Keeping track of the derivations it refers to.

In addition, the substitution in (1) must be done lazily, so that you
get the output file name for the ‘%current-system’ value when the gexp
is used.

The ‘gexp’ form is essentially like ‘quasiquote’, with ‘ungexp’ as the
counterpart of ‘unquote’:

  (gexp (system* (string-append (ungexp coreutils) "/bin/ls")))

That gexp can then be passed to ‘gexp->file’, which returns a derivation
that builds a file containing:

  (system* (string-append "/gnu/store/…" "/bin/ls"))

And it can also be used to describe derivation builds:

  (gexp->derivation "foo"
                    (gexp
                      (begin
                        (mkdir (ungexp output))
                        (chdir (ungexp output))
                        (symlink (ungexp coreutils) "cu"))))

Note that we don’t need #:inputs here, because the gexp embeds that
info.  So eventually, we could even get rid of the label in the ‘inputs’
field of packages (not a priority, though.)

We could use some sugar to make it more concise (suggestions welcome):

  (gexp->derivation "foo"
                    #~(begin
                        (mkdir #$output)
                        (chdir #$output)
                        (symlink #$coreutils "cu")))


3. Conclusion
==============

I think gexps can be helpful to serialize Scheme code that refers to
packages/derivations.  Preliminary work is in ‘wip-gexp’.
What do you think?


4. Related Work :-)
====================

HOP has a quotation mechanism to introduce client-side code in server
code and vice-versa:

  (<BUTTON>
    :onclick ~(with-hop ($foo)))

Nix uses string interpolation to keep achieve what we do:

  script = ''${coreutils}/bin/ls'';

Here ${coreutils} is replaced by the output file name, and the resulting
string internally has additional info about the reference to that
derivation.

Ludo’.

signature.asc
Description: PGP signature