Re: [gwl-devel] support for containers

[zimoun]

Hi Ricardo,

On Wed, 30 Jan 2019 at 00:16, Ricardo Wurmus <address@hidden> wrote:

> Since we don’t hash the data (because it’s expensive) the scripts are
> “proxies” for the data files.  We compute the hashes over the dependent
> scripts and assume that this is enough to decide whether to recompute
> data files or to serve them from the cache/store.

Just to be sure to well understand your point, let pick the simple
example from genomics pipeline:
 FASTQ -align-> BAM -variant-> VCF
So, you intend to hash:
 - the data FASTQ
 - the scripts align and variant
Or only the scripts containing reference to inputs (here FASTQ), where
the reference is a location fixed by the user.

Well, hashing the scripts and assuming they "mirror" the data files
appear to me an efficient design for the CAS.

--
simon