Re: [Pan-users] suggestions for large groups

[jef]

Erik wrote:
> > I was wondering if anyone could give some tips for using Pan with large
> > binary groups. I have a gig of memory in my machine, and Pan's cache is
> > set to a gig as well. However, loading a newsgroup with about 2 million
> > headers ends up causing Pan to start thrashing swap space about 2/3 of the
> > way through loading the group, or downloading that many new headers. The 
> > machine slows down to a crawl and the rest of the (down)load takes 
> > excessively long.
>
>
> This is because you have 1 gig of cache. Lower it to a reasonable
> amount and try again. 10 MB here and no high memory usage, swapping,
> etc, even on machines with 128-256MB of RAM.

	Only 10MB here, and I see the same behavior on groups this large. I 
mentioned this problem a while back, and Charles asked for a copy of the 
group's header file so that he could look into it.

	I haven't heard of any outcome from this, but the file i sent, for 
alt.binaries.multimedia, is 350MB. Sorting, scoring, etc on a file that 
big is bound to put a bit of stress on things - but I'm hoping that  the 
Pan crew can work some magic as they have in the past.

	One of the things I noted in a previous post was that Pan used to do 
this on groups with only 100,000 headers, but a tweak was made that 
improved performance dramatically at the time. I'm hoping that they are 
able to find a similar tweak for even larger groups.  (And I just got 
word recently that Giganews is increasing binary retention, so I'm 
hoping they can work it out :)

        
        jef