[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[gwl-devel] Next steps for the GWL

From: Ricardo Wurmus
Subject: [gwl-devel] Next steps for the GWL
Date: Wed, 29 May 2019 15:47:18 +0200
User-agent: mu4e 1.2.0; emacs 26.2


I’m going to use the GWL in the next few days to rewrite the PiGx RNAseq
pipeline from Snakemake.  This will likely show me what features are
still missing from the GWL and what implemented features are awkward to

My goals for the future (in no particular order) are as follows:

* add support for running workflows from a file (without

* replace the “qsub”-based Grid Engine execution engine with proper
  DRMAA bindings

* add support for running remote jobs via Guile SSH

* add support for spawning jobs on AWS or Azure or via Kubernetes

* tighter integration with Guix features, e.g. to export a container
  image per process via “guix pack” or to pack up the whole workflow as
  a relocatable executable.

* inversion of control: enable workflow designers to use the GWL as a
  library, so that the “guix workflow” user interface does not need to
  be used at all (see PiGx for an example).

* explore the use of inferiors — the GWL should be usable with any
  version of Guix that may be installed, not just the version that was
  used at compilation time.  Can we use “guix repl” and inferiors,

* add support for executing processes in isolated environments
  (containers) — this requires a better understanding of process inputs.

What do you think about this?  Who would like to help flesh out these


reply via email to

[Prev in Thread] Current Thread [Next in Thread]