Re: [Pan-devel] potential memory improvement

[kgeis]

I'm running on x64.

I was considering recompiling Pan as a 32-bit app, but I'm not convinced that I 
have all of the libraries and headers to do that.  I might have the libraries, 
so maybe I'll try something like installing a x64 package and then getting a 
binary out of an i386 package.


Ken

> -----Original Message-----
> From: Charles Kerr [mailto:address@hidden
> Sent: Sunday, May 13, 2007 04:30 AM
> To: 'Discussions of Pan source code hacking.'
> Subject: Re: [Pan-devel] potential memory improvement
>
> Ken Geis wrote:
> > I've been loading a BIG group off newsfeeds.com and my ISP, and Pan
> > (0.126) often gets up to 2G of memory usage when I visit this group.
> > I'm starting to look into the usage of STL within the Quark class.
> >
> > I'm probably not going to find big areas for improvement on memory
> > usage, but I'm going to try.
>
> :)
>
> > I think that __gnu_cxx::hashtable can be used instead of
> > __gnu_cxx::hash_map for Quark._lookup.  That could save the size of a
> > StringView for each entry in the hashtable.  hash_map is redundant
> > because in each node, it stores a key-value pair; the value that Quark
> > is storing (Impl) already contains the key.  I'm looking in to this.  If
> > anyone else wants to look into it, it'd be nice.  I haven't touched C++
> > in several years.
>
> This is a good idea, and I've spent a couple of minutes dinking with it.
> hash_set looks like a very thin wrapper around hashtable, and still has
> a non-gcc parallel (std::set) that we can fall back on.
>
> Suprisingly, hash_set and hash_table had nearly identical memory use.
> Also, sizeof(Impl) stubbornly stayed at 24 on my x64 box even after
> I changed StringView::len and Impl::refcount from ulongs to uint32_ts.
>
> By bypassing StringView altogether and having defining Impl to
> hold a char* and two uint32_t's for len and refcount, I was able
> to get sizeof(Impl) down to 16.  Top said that in a large group
> this saved about 5 megs of memory and loaded articles about 15%
> faster, so it's worth keeping as a first stab at the problem,
> but it doesn't fix the 2GB kind of overhead you're seeing.
>
> Incidentally, are you on an 32-bit or 64-bit architecture?
>
> Charles
>
>
> _______________________________________________
> Pan-devel mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/pan-devel
>