Re: sort -m does too much work

[Bo Borgerson]

David Muir Sharnoff wrote:
> I've got 200 1GB pre-sorted files.   If I try to merge
> them with sort -m, it is obvioulsy trying to do too much
> work: after running for a couple minutes, it has not
> produced any output but it has made a 5 GB temporary
> file.
> 
> When the input is pre-sorted, no temporary file should
> be required.
> 
> Output should begin immediately.


Hi David,

The reason you're not seeing output immediately is because sort
internally limits the number of files it will read at once.  By default
this limit is set to 16.  When more files are to be merged, sort uses
temporary files.

Starting in release 7.0 this limit will be modifiable on the
command-line using the --bath-size=N option.  With 200 files you'll
still need to balance your desire for immediate output against the
performance implications of reading from so many files at once, but the
choice will be yours.

Thanks,

Bo