Re: some curious command-line garbling using gnu parallel (?)

[Ole Tange]

On Thu, Apr 7, 2011 at 2:56 PM, Jesse Alama <jesse.alama@gmail.com> wrote:
> Hi Ole,

There are lot of capable users on the mailinglist. They may be able to
help you, too.

> I'm not sure what the source of the trouble is here, but since it involves
> GNU parallel, I thought I'd ask for your advice.
>
> Here is an invocation of parallel that gives strange results:
>
> =======================================================================
> $ find . -name "ckb*.xml1" ! -empty | parallel --eta --jobs +0 "xsltproc
> --param colored '1' --param proof_links '1' --param linking 's' --param
> ajax_proofs '1' ~mizar-items/xsl4mizar/MHTML/mhtml_main.xsl {} > {.}.html 2>
> /dev/null"

Start by disabling --eta. It is known that --eta causes problems. We
still need to find a reproducible error, but we know it messes up
stuff.

Next: Create an error that I can reproduce on my system. I do not have
~mizar-items/xsl4mizar/MHTML/mhtml_main.xsl or any of the ckb files.
So create a tar file of those or adjust the command so it does not
depend on that.

> Computers / CPU cores / Max jobs to run
> 1:local / 24 / 24
> sh: cannot create ordan4/text/ckb16.html: Directory nonexistent
> sh: cannot create bel_2/text/ckb76.html: Directory nonexistent
> sh: cannot create lor_2/text/ckb17.html: Directory nonexistent
>
> Computer:jobs running/jobs completed/%of started jobs/Average seconds to
> complete
> ETA: 162097s 69819left 0.02avg  local:24/3195/100%/0.0s sh: cannot create
> ./cat_4/text/sp_1/text/ckb26.html: Directory nonexistent
> ETA: 1644s 69413left 0.02avg  local:24/3601/100%/0.0s sh: cannot create
> ./convfun1/text/ckbyalg1/text/ckb15.html: Directory nonexistent
> ETA: 1646s 69279left 0.02avg  local:24/3735/100%/0.0s sh: cannot create
> ./jtdlat_2/text/ckb37.html: Directory nonexistent
>
> ...
> =======================================================================
>
> I'm trying to use xsltproc to apply a stylesheet to a large number of XML
> files.  What is strange is that xsltproc (or possibly gnu parallel) is
> receiving strange argument.  For example, the first error above
>
> sh: cannot create ordan4/text/ckb16.html: Directory nonexistent
>
> shows that a path is getting mangled: the path should be
>
>  ./jordan4/text/ckb16.html
>
> not
>
>  ordan4/text/ckb16.html
>
> Similarly, looking at the second example,
>
> sh: cannot create bel_2/text/ckb76.html: Directory nonexistent
>
> I can say that the path should be
>
>  waybel_2/text/ckb76.html
>
> not
>
>  bel_2/text/ckb76.html

Yep, it seems {.} somehow does something wrong.

> All the errors above have this flavor: some initial part of the path is
> getting omitted.  I don't know what's going on.  Perhaps find is passing
> strange paths to parallel (in which case this is not parallel's fault).  Or
> perhaps something is wrong with the pipe that passes the arguments to
> parallel.  Or perhaps gnu parallel is itself doing something strange with
> the paths.
>
> Do you have any ideas?

Try without --eta.

Try -j1 to see if it works when running in serial.

> Best regards,
>
> Jesse
>
> --
> Jesse Alama
> http://centria.di.fct.unl.pt/~alama/

/Ole