[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: NMERGE hardcoded in sort.c

From: Andrew D Jewell
Subject: Re: NMERGE hardcoded in sort.c
Date: Wed, 1 Oct 2003 12:59:10 -0400

You can get our version of coreutils 4.5.3 from http://alexautils.sourceforge.net/

We've had good results with NMERGE up around 1000

The merge code in standard sort is O(NMERGE * total_lines) which is already bad at 16 and terrible if you get much higher. The alexautils code has different merging code which is O(log(NMERGE) * total_lines) which works great.


At 12:57 PM +0200 9/30/03, Paul Courbis wrote:

I'm using "sort" to, surprisingly, sort huge files (up to 2 GB). Of course I had to use the -S switch to tune memory usage. My concern is that I don't have enough memory to sort in RAM, thus sort is using temporary files tat it merges to get the final result.

Having a look to sort's sources I noticed that NMERGE is hardocoded (it's not even an option).

My questions are :

- I did a patch to make sort accept a new option (currently called -N to setup NMERGE). Is someone interested by this patch ? - I'm doing some benches, but I already noticed that 16 (the hardcoded value) is not always the best one. For example my first serie of benches showed me that 18 is better in this particular case. Did someone worked on a model allowing to guess what would be the best value for NMERGE for a given set of data to sort ?

Any hint would be apreciated



Bug-coreutils mailing list

reply via email to

[Prev in Thread] Current Thread [Next in Thread]