parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Parallel grep example


From: Ole Tange
Subject: Re: Parallel grep example
Date: Wed, 5 Dec 2012 14:52:42 +0100

On Mon, Dec 3, 2012 at 11:36 PM, Harry B <address@hidden> wrote:
> Hello,
>
> I tried the parallel grep example on a sample file  (1.4G and 3,153,199
> lines). The parallel grep
> (http://www.gnu.org/software/parallel/man.html#example__parallel_grep)  is
> considerably slower - I am trying to see where the bottle neck is.  Is there
> an easy way for me to guess which way would be better based on the size of
> the file?

GNU Parallel is the bottleneck. It can only move 50-100 MB/s in --pipe mode.

So while the example without --pipe may speed up your grep, then the
second part with --pipe most likely will not. That should probably be
stressed in the documentation that the second part is only to show how
it can be done, and not that it will actually run faster.


/Ole



reply via email to

[Prev in Thread] Current Thread [Next in Thread]