Re: [bug-gawk] [BugReport] GAWK 3.1.8 / gsub() memory leak

[Koehler, Mario]

Hi,

thanks for getting back to me. I have to admit I didn't try out the 4.x version 
before I sent my report. Sorry for that.

To have a quick fix we went back to 3.1.6 for now, which we found out does also 
not have the bug. But it's good to know that we can upgrade to 4.x.

Thanks again!

Best regards
Mario


-----Ursprüngliche Nachricht-----
Von: Aharon Robbins [mailto:address@hidden 
Gesendet: Donnerstag, 29. November 2012 22:25
An: Koehler, Mario; address@hidden
Betreff: Re: [bug-gawk] [BugReport] GAWK 3.1.8 / gsub() memory leak

Greetings. Re: this:

> From: "Koehler, Mario" <address@hidden>
> To: "'address@hidden'" <address@hidden>
> Date: Thu, 29 Nov 2012 09:29:29 +0100
> Subject: [bug-gawk] [BugReport] GAWK 3.1.8 / gsub() memory leak
>
> Hi,
>
> i would like to report a memory leak in gensub() and gsub() functions 
> of GAWK 3.1.8. The bug seems to be introduced somewhere between 3.1.5 
> and 3.1.8. Those two versions are the ones I have access to, and in 
> 3.1.5 the bug does not occur, but in 3.1.8 it does.
>
> The memory leak scenario can be reproduced with this small AWK script:
>
> {
>     outputLineTemplate = "COMP_ID-KKTNR-KUKTO|CAP_NAME|CAP_STR 
> KBHNR|PADDED_PLZ|CAP_ORT|BIRTHDAY|256,COMP_NR|TELEPHONE";
>     tokenList = "COMP_NR COMP_ID KKTNR KUKTO CAP_NAME BIRTHDAY CAP_STR KBHNR 
> PADDED_PLZ CAP_ORT TELEPHONE"
>     tokenCount = split(tokenList, definedTokens, " ");
>
>     while (1 > 0) {
>          outline = outputLineTemplate;
>          for( token = 1; token <= tokenCount; token++ ) {
>            gsub(definedTokens[token],"example value",outline);
>          }
>          print outline;
>     }
> }
>
> The script defines a "template" for an output line that contains 
> placeholders. Then it iterates over this output line template and 
> replaces each placeholder (token) in the template with a certain value 
> ("example value" in this case). This is repeated in an infinite loop 
> to simulate the processing of a large batch of input lines. The script 
> was originally part of a much larger "conversion script" and this 
> relevant part was extracted from there and slightly modified for the purpose 
> of this bug report.
> 
> The most important line is the line with the gsub() call (replacing it 
> with gensub() also produces the bug). This call causes a memory leak 
> to occur, which can be easily observed by starting the script and then 
> watch "top" for a while. You will see that the memory usage of the awk 
> process will rise continually.
> 
> If you repeat this test with GAWK 3.1.5 you will see that the memory 
> usage of the awk process will remain constant at a very low level 
> (couple of KB)
>
> I did my testing on SUSE Linux Enterprise Server 11.2 (x86_64).
>
> Best regards
> Mario K?hler

Thank you for taking the time to send in a bug report. I appreciate your 
concern.

I can indeed reproduce this.

Fortunately, the bug has been fixed in subsequent versions. The current 
released version is 4.0.1 and it does not display the bug.  It's not obvious to 
me (now) what fixed the bug, but the 'gawk-stable' branch in git, which has the 
few additional changes over 3.1.8, does not have the bug either.

I suggest you simply download and install the current version. Cut & Paste:

        wget http://ftp.gnu.org/gnu/gawk/gawk-4.0.1.tar.gz
        tar -xpzf gawk-4.0.1.tar.gz
        cd gawk-4.0.1
        ./configure && make && make check
        sudo make install

:-)

Thanks,

Arnold

P.S. If you can get Suse to upgrade, all the better.