Re: need help related to gnu parallel

[Ole Tange]

On Thu, Jul 26, 2012 at 2:44 AM, Rohit Awasthi <rohitawas@gmail.com> wrote:
> Hi everyone, I need some help with shell scripting.
>
> I have shell script which in the for loop goes through about 1000 hosts and
> downloads files using wget
>
> Example:
>
> for i in hosts.file
> do
> time wget>> log
> done
>
> Now this process takes so long to go through so many hosts on the network,
> and what I would like to do is to run about 10 or 20 of those commands
> concurrently in parallel, so instead of hours, it would take much quicker to
> get data collected.
> I am using gnu parallel for this purpose, but not able to do it.

Please confirm that:

* you have watched https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
* you have read all the examples in the man page
  
http://www.gnu.org/software/parallel/man.html#example__working_as_xargs__n1__argument_appending
* you have read about the option --nonall

After you have confirmed that, I will be happy to help you out.


/Ole