[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk
From: |
Andrew J. Schorr |
Subject: |
Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4 |
Date: |
Wed, 28 Aug 2019 15:04:58 -0400 |
User-agent: |
Mutt/1.5.21 (2010-09-15) |
On Wed, Aug 28, 2019 at 02:56:55PM -0400, Andrew J. Schorr wrote:
> I have to imagine that there's a reference counting issue with the
> $1 field arising from 'mom_version !~ $1'. I see this in the
> valgrind output when not setting fixram (note that purge_record never
> appears in the valgrind log in the fixram=1 case):
So I patched field.c like so:
diff --git a/field.c b/field.c
index feba8fd..94614a6 100644
--- a/field.c
+++ b/field.c
@@ -361,6 +361,7 @@ purge_record()
memcpy(r->stptr, save, r->stlen);
r->stptr[r->stlen] = '\0';
r->flags |= MALLOC;
+ printf("debug: purge_record saving field %d with refcnt
%ld\n", i, r->valref);
}
unref(r);
getnode(n);
Then I ran:
bash-4.2$ ./gawk -v fixram=1 -f /extra_disk/tmp/gbug/simple_mem.awk
/extra_disk/tmp/gbug/sqlRefMap.txt
RSS per record = 0.13
All OK. But without fixram=1:
RSS per record = 0.13
bash-4.2$ ./gawk -f /tmp/simple_mem.awk /tmp/sqlRefMap.txt
...
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
debug: purge_record saving field 1 with refcnt 2
RSS per record = 0.13
The test script is attached.
Regards,
Andy
simple_mem.awk
Description: Text document
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, (continued)
Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, arnold, 2019/08/25
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Finn Magnusson, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, arnold, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4,
Andrew J. Schorr <=
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/28
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, arnold, 2019/08/29
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, Andrew J. Schorr, 2019/08/29
- Re: [bug-gawk] Increased RAM memory usage in gawk 5.0.1 compared to gawk 4.1.4, arnold, 2019/08/29
- Re: [bug-gawk] memory issues, Andrew J. Schorr, 2019/08/29