bug#43389: 28.0.50; Emacs memory leaks

[Eli Zaretskii]

> Date: Tue, 15 Sep 2020 23:12:09 +0200
> From: Russell Adams <RLAdams@AdamsInfoServ.Com>
> 
> > Can you use some utility that produces a memory map of an application,
> > and see how much of those 5GB are actually free for allocation by
> > Emacs?
> 
> Any suggestions?

Your Internet search is as good as mine.  This page offers some
possibilities:

  
https://stackoverflow.com/questions/36523584/how-to-see-memory-layout-of-my-program-in-c-during-run-time

> > Also, do you see any libraries used by Emacs that have high
> > memory usage?
> 
> Emacs is the top memory usage on my laptop, firefox is second at
> 2GB. The rest are <1G.

No, I meant the shared libraries that Emacs loads.  Maybe one of them
has a leak, not Emacs's own code.

> > 28% and 33% of what amount?
> 
> 16GB
> 
> > If your RSS is 5GB after 4 days of uptime, and the memory footprint
> > grows at a constant rate, it would mean more than 1GB per day.  But
> > I'm guessing that 33% - 28% = 5% of your total memory is much less
> > than 1GB.
> 
> No, 33% is ~5GB. ;]
> 
> > In which case the memory footprint must sometimes jump by
> > very large amounts, not grow slowly and monotonically each day.
> > Right?  So which events cause those sudden increases in RSS?
> 
> I can't say.

Well, actually the above seems to indicate that your memory footprint
grows by about 1GB each day: 5% of 16GB is 0.8GB.  So maybe my guess
is wrong, and the memory does increase roughly linearly with time.
Hmm...

We had a discussion several times regarding the possible effects of
the fact that glibc doesn't return malloc'ed memory to the system.  I
don't think we reached any firm conclusions about that, but it could
be that some usage patterns cause memory fragmentation, whereby small
chunks of free'd memory gets "trapped" between regions of used memory,
and cannot be reallocated.

We used to use some specialized malloc features to prevent this, but
AFAIU they are no longer supported on modern GNU/Linux systems.

Not sure whether this is relevant to what you see.

Anyway, I think the way forward is to try to understand which code
"owns" the bulk of the 5GB memory.  Then maybe we will have some
ideas.

Thanks.