Hi,
I have a large file of some patterns and need to grep & find other associated things for every pattern in another large file.
So i did some thing like this -
cat ./bigfilepat.sort |parallel -j +17 'grep {} ./bigfilematch.txt |/home/username/processer.pl'
processor.pl does some parse/format & echo to stdout thing only.
But at anytime when I do a ps aux |grep parallel |grep bigfile I see max 4/5 & min 1 programs running only.Why is this so ? And also it take long long time to complete.
What is the best way to solve this problem ? Thanks in advance.
Ciao.