Re: [GWL] (random) next steps?

[Ricardo Wurmus]

Hi simon,

> Here, I would like to collect some discussions or ideas about the Guix
> Workflow Language (GWL) and the next steps of this awesome tool!

thanks for kicking off this discussion!

> With wisp, the workflow description seems close to CWL, which is an argument 
> ;-)
> Last time, I check, CWL files seems flat yaml-like files and they lack
> programmable extension; as Snakemake provides with Python for example.
> And GWL-wisp will have both: easy syntax and programmable stuff,
> because it is still lisp under the hood.

I’m working on updating the GWL manual to show a simple wispy example.

> 4.
> Some GWL scripts are already there.
> Could we centralize them to one repo?
> Even if they are not clean. I mean something in this flavor:
> https://github.com/common-workflow-language/workflows

I only know of Roel’s ATACseq workflow[1], but we could add a few more
independent process definitions for simple tasks such as sequence
alignment, trimming, etc.  This could be a git subtree that includes an
independent repository.

[1]: https://github.com/UMCUGenetics/gwl-atacseq/

> 5.
> I recently have discovered the ELisp package `s.el` via the blog post:
> http://kitchingroup.cheme.cmu.edu/blog/2018/05/14/f-strings-in-emacs-lisp/
> or other said:
> https://github.com/alphapapa/elexandria/blob/master/elexandria.el#L224
>
> Does it appear to you a right path to write a "formater" in this
> flavour instead of the `string-append` ?
> I mean, e.g.,
>   `(system ,(string-command "gzip  ${data-inputs}  -c >  ${outputs}"))
> instead of e.g.,
>   `(system ,(string-append "gzip " data-inputs " -c > " outputs))
>
> It seems more on the flavour of  Snakemake.

Scheme itself has (format #f "…" foo bar) for string interpolation.
With a little macro we could generate the right “format” invocation, so
that the user could do something similar to what you suggested:

    (shell "gzip ${data-inputs} -c > ${outputs}")

    –> (system (format #f "gzip ~a -c > ~a" data-inputs outputs))

String concatenation is one possibility, but I hope we can do better
than that.  scsh offers special process forms that would allow us to do
things like this:

    (shell (gzip ,data-inputs -c > ,outputs))

or

    (run (gzip ,data-inputs -c)
         (> 1 ,outputs))

Maybe we can take some inspiration from scsh.

> 6.
> The graph of dependencies between the processes/units/rules is written
> by hand. What should be the best strategy to capture it ? By files "à
> la" Snakemake ? Other ?

The GWL currently does not use the input information provided by the
user in the data-inputs field.  For the content addressible store we
will need to change this.  The GWL will then be able of determining that
data-inputs are in fact the outputs of other processes.

> 7.
> Does it appear to you a good idea to provide a command `guix workflow pack` ?
> to produce an archive with the binaries or the commit hashes of the
> channels, etc.

This shouldn’t be difficult to implement as all the needed pieces
already exist.

> Last, the webpage [1] points to gwl-devel mailing list which seems broken.
> Does the gwl-devel need to be activated and the GWL discussion will
> append there or everything stay here to not scattered too much.
>
> [1] https://www.guixwl.org/community

Hmm, looks like the mailing list exists, but has never been used.
That’s why there is no archive.  Let’s see if this email creates this
archive.

--
Ricardo