Re: [PATCH] Improve memory management in join

[Jim Meyering]

"Bo Borgerson" <address@hidden> wrote:
> This improves the performance of `join' by reducing memory management
> overhead and eliminating unnecessary copies for order checking:
>
> $ valgrind src/join.master ja jb
> ==23744== malloc/free: 4,571,152 allocs, 4,571,152 frees, 255,971,774
> bytes allocated.
>
> $ valgrind src/join ja jb
> ==23738== malloc/free: 1,405 allocs, 1,405 frees, 65,858 bytes allocated.
>
> $ time src/join.master ja jb
> user    0m27.126s
>
> $ time src/join ja jb
> user    0m17.297s

Oooh.  That's nice.
Hard to argue with such results.

What do you think about making the above into coreutils' first
performance-measuring test?
It would run only if valgrind is available, sizeof void* == N
(for whatever N is on some type of system), and that would fail
if the total number of bytes allocates is larger than say, 70000?

I'll look at the patch itself in the next few days.

> Subject: [PATCH] Improve memory management in join
>
> * src/join.c (struct seq): Use a (struct line **) for `lines' rather than
> one long (struct line *).  This allows individual lines to be swapped out
> if necessary.
> (reset_line): Get a line ready for new input.
> (init_linep): Create a new line and assign it to the the pointer passed in.
> (spareline[2]): Hold a spare line for each input file.
> (free_spareline): Clean up.
> (get_line): Take a (struct line **) instead of a (struct line *).  If the
> line to be overwritten is the previous line for the current file then swap
> it out for the spare.
> (join): Accomodate new structure of SEQs and new parameters to get_line;
> Don't free stale lines until the end -- they're re-usable now.
> (dup_line): Removed.