Re: Parallel a script

[Ole Tange]

On Sun, Jun 26, 2011 at 12:43 AM, yacob sen <yacob_123@yahoo.com> wrote:
>
> Dear Ole
> Thank you very much, really, Ole.

If you like GNU Parallel:
* Post the intro videos on forums/blogs/Twitter/Facebook/Linkedin
* Request or write a review for your favourite blog or magazine
* Request or build a package for your favourite distribution (if it is
already there, send a "thank you" note to the maintainer)
* Invite me as a speaker for your next conference

If GNU Parallel saves you money:
* (Have your company) donate to FSF https://my.fsf.org/donate/

> Now, the program works the way it is supposed to. I compared a serial version 
> and the parallel version. I have time stamp for the serial using the "time" 
> command :
> real 6m33.524s
> user 4m58.650s
> sys 0m16.050s
>
> while for the parallel
> Computers / CPU cores / Max jobs to run
> 1:local / 4 / 4
> real 2m43.622s
> user 7m33.660s
> sys 0m21.950s
> I used also python parallel processing and I suppose "time.time" command in 
> python does print the "real" time as the "time" command in Linux which came 
> to 2m56.842s a little worse than the GNU parallel.
> I run my program on 4 files using all the 4 cores (-j+0 option) for the above 
> example. Again I used the the GNU parallel on 8 files. I got a time stamp of :
> real 4m56.619s
> user 14m7.820s
> sys 0m42.640s
>
> I am not sure how the GNU parallel works. But I think it acts on the first 4 
> files first sending each instances of the program acting on each file and as 
> soon as the process frees itself the program continues to the rest of the 
> file and so on until all the files are dealt with ?
> I would like to hear how it works exactly ?

-j gives the number of workers. Each worker gets one job from the
queue and works on that. When a worker finishes he is given a new job
from the queue until the queue is empty. When the queue is empty GNU
Parallel waits for all workers to finish before finishing.

Most of this is actually covered in the intro videos. Did you watch those?

http://www.youtube.com/watch?v=OpaiGYxkSuQ
http://www.youtube.com/watch?v=P40akGWJ_gY
http://www.youtube.com/watch?v=1ntxT-47VPA
http://www.youtube.com/watch?v=fOX1EyHkQwc

> Kindest Regards
> Yacob

/Ole