Re: Parallel grep example

[Ole Tange]

On Mon, Dec 3, 2012 at 11:36 PM, Harry B <harrysungod@gmail.com> wrote:
> Hello,
>
> I tried the parallel grep example on a sample file  (1.4G and 3,153,199
> lines). The parallel grep
> (http://www.gnu.org/software/parallel/man.html#example__parallel_grep)  is
> considerably slower - I am trying to see where the bottle neck is.  Is there
> an easy way for me to guess which way would be better based on the size of
> the file?

GNU Parallel is the bottleneck. It can only move 50-100 MB/s in --pipe mode.

So while the example without --pipe may speed up your grep, then the
second part with --pipe most likely will not. That should probably be
stressed in the documentation that the second part is only to show how
it can be done, and not that it will actually run faster.


/Ole