|
From: | Blake, Lauren |
Subject: | Help on For loop Rewrite |
Date: | Mon, 12 May 2014 15:21:49 +0000 |
Hello,
I was hoping to get some guidance on a mistake that I am making in rewriting a for loop to perform the iterations of the loop in parallel using GNU Parallel. I have changed the for loop to the set-up that I found on many examples online, but I'm afraid
that the syntax is not correct still, and I am still getting errors.
When running my current edited script, I am getting errors saying "no file/directory *Name of file* exists" meaning that the commands that are supposed to create these files and directories are not being run properly. This leads me to believe that lines
15 - 19 of my edited (parallel) script For_loop_parallel.sh (attached) are not functioning properly.
On the man page, I examined the example they provided:
for x in `cat list` ; do no_extension=${x%.*}; do_something $x scale $no_extension.jpg do_step2 <$x $no_extension done) | process_output and how they changed it to this: cat list | parallel "do_something {} scale {.}.jpg ; do_step2 <{} {.}" | process_output But in that example, I do not understand how the parallel command will "know" to iterate through the x variable, as that disappeared in the parallel version.
I believe my problem in my rewritten script just stems from my lack of understanding of how to tell GNU parallel what to iterate from. In my for loop, I am trying to iterate through each line using variable $line through a file called $fName (the Looped_through_file.txt),
but currently don't see a way of including that in proper syntax for the parallel command. If you open up both of the For_loop scripts, you will see how I attempted to change the syntax of the (fully functioning) for_loop_regular script into the for_loop_parallel
script based on the online examples.
I did not include the large file that one of the commands references, as it is extremely large, but I did include the small file that is being referenced and is actually being looped through.
Any advice on properly rewriting my for loop so that the subsequent iterations can be sent to different cores will be very helpful. I have yet to see an example that maintains the iterated variable in the parallel command and do not understand how parallel
can work without that information. However, I am new to Linux commands and programming in general, so perhaps there is some large aspect of Linux capabilities/syntax that I am unaware of.
Thank you,
Lauren
|
For_loop_parallel.sh
Description: For_loop_parallel.sh
For_loop_regular.sh
Description: For_loop_regular.sh
Looped_through_file.txt
Description: Looped_through_file.txt
[Prev in Thread] | Current Thread | [Next in Thread] |