Re: --transfer with absolute path does not respect --workdir

[Ole Tange]

On Fri, Jun 15, 2012 at 2:29 PM, Jay Hacker <jayqhacker@gmail.com> wrote:
> [Moving this from bug tracker to mailing list at Ole's request]
>
> Currently, if you use --sshlogin to run jobs on remote computers and
> --transfer to send files, the files are placed in your home directory,
> including any paths.  So, as the man page says:
>
>    echo foo/bar.txt | parallel --sshlogin server.example.com --transfer wc

The idea of --transfer is that {} will be replaced with the path to the file.

So this will work:

    echo foo/bar.txt | parallel --sshlogin server.example.com --transfer wc {}
    echo /foo/bar.txt | parallel --sshlogin server.example.com --transfer wc {}

The first will transfer the file with the relative path to the path
relative to your home dir and run:

  wc foo/bar.txt

The second will transfer the file to /foo/bar.txt and run:

  wc /foo/bar.txt

I would find it very surprising if the above did not work if I chose
to use absolute paths instead of relative paths, and I would find it
very surprising if {} was suddenly replaced with something else than
what was given as input, and that is the problem if absolute source
paths should be transferred to relative destinations paths.

Assume /foo/bar.txt is instead transferred to ~/foo/bar.txt. Then
replacing {} with /foo/bar.txt:

  wc /foo/bar.txt

will not work. If we want that to work then {} has to be replaced with
./foo/bar.txt or foo/bar.txt. And then we would replace with something
different from

If /foo/bar.txt was transferred to ~/foo/bar.txt then {} would not
point to the file.

> a.) surprising, because --transfer foo/bar.txt and $PWD/foo/bar.txt do
> very different things,

If your home dir is the same on both sending and receiving server then
those will be experienced almost identical to the user.

> So for example:
>
> $ parallel -S localhost --transfer ls ::: /etc/issue
>
> Should give etc/issue in your $HOME, and

How? Are you going to change the replacementstring for {} (from
/etc/issue to etc/issue)?

> $ parallel -S localhost --workdir /tmp --transfer ls ::: /etc/issue
>
> should give /tmp/etc/issue.

How? Are you going to change the replacementstring for {} (from
/etc/issue to /tmp/etc/issue)?

If you change the replacementstring what should these then do:

$ parallel -S localhost --workdir /tmp ls ::: /etc/issue
$ parallel -S localhost ls ::: /etc/issue
$ parallel -S localhost --transfer ls ::: foo/bar.txt
$ parallel -S localhost --transfer ls ::: $PWD/foo/bar.txt

If I wanted to --transfer from an absolute path to an absolute path,
how would I do that using your syntax?


/Ole