parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

newbie needs some help with setup


From: Chris
Subject: newbie needs some help with setup
Date: Wed, 04 Mar 2015 02:51:31 -0800
User-agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.5.0

Hello,

First time using GNU parallel. I hope it is the right tool for the job, and that you might give some advice :)

Here is the job:

- 350 independent php data processing scripts, run every 8 hours. Uses up to 100MB RAM per process, but most will use only 10-20MB. Scripts run 1-90 minutes.
example: php crons.php -run_job=allstarhealth > allstarhealth.txt 2>&1

- Amazon EC2 Linux Fedora instance. VM with one core, one CPU, and 1G memory

- GOAL: launch the php scripts in parallel on a single core, and remain below 10% CPU (AWS Free Tier) and below 1GB RAM utilization.

My question is, what are the appropriate options to suit that goal? Based on the docs, this is what I'm guessing. Please give your opinion!:

# parallel \
    --max-procs N \
    --semaphore \
--load 0.1 \ # not sure if this is for CPU or RAM, assume CPU. Need to "wait until CPU <= 9%" before launching next process
    --memfree size \    # useful?
--noswap \ # we do have a swap file, but performance is very slow
   -a file_list.sh


thanks in advance



reply via email to

[Prev in Thread] Current Thread [Next in Thread]