Re: Parallelizing sort

[Glen Lenker]

On Wed, Feb 11, 2009 at 10:12:42PM -0800, Nima Nikzad wrote:
> Hello coreutils community,
> I am new to the exciting world of open source development and thought I
> should introduce my self and get some feedback on what I am working on.  My
> name is Nima Nikzad and I, along with a small group of students, am working
> on changes to coreutils as part of a project for Prof. Paul Eggert's
> software engineering course at UCLA.  Specifically, we are looking at how we
> can better utilize modern multi-core processors when sorting and merging
> files.  My group is first taking a look at how to parallelize the external
> sort phase of sort's operation.  Another group of students is taking a look
> at in-memory sort while we are working on this.
> 
> Has anyone already tried their hand at parallelizing this code?  Are there
> any special considerations we should keep in mind while moving forward?  My
> group was looking to tackle the problem by using OpenMP as we have some
> experience working with it in the past and like that we can do quite a bit
> with it while (hopefully) having little impact on the structure of
> the existing code.  Does anyone have any feedback on threading technologies
> that would be appropriate for this project or does anyone think OpenMP is a
> poor choice?
> 
> I look forward to hearing your suggestions!  We are looking to have
> something implemented and benchmarked soon and I will be sure to keep in
> contact with the community.  Please feel free to reach me at nnikzad at
> ucla.edu.  Thank you!
> 
> - Nima Nikzad

Hi everyone,

My name is Glen Lenker. I am the project leader for the second group
working to parallelize sort under Paul Eggert.

As mentioned above, my group is primarily focused on parallelizing the
"in-memory" sort, but at the moment we are still considering working to
parallelize other aspects of the current sort as well. At the moment we
are considering using the Gnulib thread module as our threading library.

Jim: We heard from Paul Eggert today that you have looked into how sort
would benefit from parallelization. I am particularly interested in your
approach. If you don't mind, perhaps you could start the discussion on
this.

-- 
Glen Lenker