Re: gsort problem (TAKE 2)

[Paul Eggert]

"Anthony Tiemens" <address@hidden> writes:

> debug output of binaary created with "-g" compile option.
>
> /CML/medw/tmp/orasupptest_gsort> gdb ./gsort-HPUX-5.94-6.7.06
> HP gdb 3.1 for PA-RISC 1.1 or 2.0 (narrow), HP-UX 11.00.
> Copyright 1986 - 2001 Free Software Foundation, Inc.
> Hewlett-Packard Wildebeest 3.1 (based on GDB) is covered by the
> GNU General Public License. Type "show copying" to see the conditions to
> change it and/or distribute copies. Type "show warranty" for warranty/support.
> ..
> (gdb) b xalloc_die
> Breakpoint 1 at 0x125b8: file xalloc-die.c, line 38.
> (gdb) r -T . -k 5n,5n -t ";" bigfile.txt -o bigfilesorted.txt
> Starting program: /CML/medw/tmp/orasupptest_gsort/./gsort-HPUX-5.94-6.7.06 -T 
> . -k 5n,5n -t ";" bigfile.txt -o bigfilesorted.txt
> ls
>
> Breakpoint 1, xalloc_die () at xalloc-die.c:38
> 38      xalloc-die.c: No such file or directory.
> (gdb) ls
> Undefined command: "ls".  Try "help".
> (gdb) where
> #0  xalloc_die () at xalloc-die.c:38
> #1  0x19244 in xnrealloc_inline (p=0x0, n=5728, s=1) at xmalloc.c:75
> #2  0x19298 in xrealloc (p=0x77e7ec90, n=5728) at xmalloc.c:91

Thanks.  It appears that realloc(0x77e7ec90, 5728) failed (i.e.,
returned NULL).  So realloc thought that memory was exhausted even
though 'sort' was asking for only 5728 bytes, which is a small number
these days.  That is a bit surprising.

Are you using the standard memory allocation routines
malloc/realloc/etc., or some unusual library?  If you are using the
standard library, can you try rebuilding it with (say) mpatrol
<http://directory.fsf.org/all/mpatrol.html> and see whether that
isolates the problem?  My understanding is that you need not recompile
'sort' to use mpatrol.  (mpatrol is not as good as valgrind, but it is
better than nothing.)

> (gdb) print merge_buffer_size
> $1 = 185835729

Wow.  That is a huge number.  It indicates that there is at least one
line of text in your file that is 185 MB long.  Or, that there is a
bug in 'sort' or in your compiler or debugger or whatnot.

Can you check all the lines in your file again?  Something like this
should do it:

gawk '{if (maxlen < length) maxlen = length} END { print maxlen}' bigfile.txt