[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Managing data files in workflows
From: |
Konrad Hinsen |
Subject: |
Re: Managing data files in workflows |
Date: |
Fri, 26 Mar 2021 16:36:08 +0100 |
Ricardo Wurmus <rekado@elephly.net> writes:
> In my test case this file exists:
>
>
> /tmp/gwl/lf6uca7zcyyldkcrxn3zwc275ax3ip676aqgjo75ybwojtl4emoq/data/weekly-incidence.csv
And that's the one that causes the error message for me when I run the
workflow a second time (see below). But as I understand now, the mistake
happens earlier, as this step shouldn't be executed at all.
> At least that happens in my case. I wonder why it doesn’t work in your
> case.
Is there anything I can do to debug this?
> Yes, that’s the way it’s supposed to work already. GWL computes the
> hashes of each chain of processes, which includes the generated process
> script, its inputs, and the hashes of all processes that lead up to this
> process. Any change in the chain will lead to a new hash and thus a
> cache miss, leading GWL to recompute.
Excellent, that's what I was hoping to happen, given the Guix
background.
Cheers,
Konrad.
$ guix workflow run test.w
info: Loading workflow file `test.w'...
info: Computing workflow `influenza-incidence'...
run: Executing: /bin/sh -c
/gnu/store/km977swwhqj2n1mg34fq6sv4a41iabkm-gwl-download.scm '((inputs)
(outputs "./data/weekly-incidence.csv") (values) (name . "download"))'
--2021-03-26 13:55:22-- http://www.sentiweb.fr/datasets/incidence-PAY-3.csv
Resolving www.sentiweb.fr (www.sentiweb.fr)... 134.157.220.17
Connecting to www.sentiweb.fr (www.sentiweb.fr)|134.157.220.17|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/csv]
Saving to: ‘./data/weekly-incidence.csv’
./data/weekly-incid [ <=> ] 83,50K --.-KB/s in 0,008s
2021-03-26 13:55:22 (10,5 MB/s) - ‘./data/weekly-incidence.csv’ saved [85509]
Backtrace:
6 (primitive-load "/home/hinsen/.config/guix/current/bin/guix")
In guix/ui.scm:
2164:12 5 (run-guix-command _ . _)
In srfi/srfi-1.scm:
460:18 4 (fold #<procedure 7f33df87ea80 at gwl/workflows.scm:388:2 (ite…> …)
460:18 3 (fold #<procedure 7f33df87ea60 at gwl/workflows.scm:391:13 (pr…> …)
In gwl/workflows.scm:
392:21 2 (_ #<process download> ())
In srfi/srfi-1.scm:
634:9 1 (for-each #<procedure 7f33df87e460 at gwl/workflows.scm:589:28…> …)
In guix/ui.scm:
566:4 0 (_ system-error "symlink" _ _ _)
guix/ui.scm:566:4: In procedure symlink: File exists:
"/tmp/gwl/mwmeuuhnu7sv4mpouj7o5x4se4qp5n5auzhpkb7y7oxidoxzc6ra/./data/weekly-incidence.csv"