Re: [Pan-users] pan for Windows crashes when reading large newsgroup

[K Shen]

Hi Steve and Duncan,

Thanks for your replies. I had hope that the 32 bit pan would be able to use 
all 4G of memory on my 64 bit Win 7 OS, but from what Steve is saying, I guess 
it is still limited to the 2G limit. (As Steve said, I don't think there is a 
64 bit pre-built binary version of pan for Windows).

I was also trying to see if there might be other reasons for the recent 
increase in memory usage; I guess the only reason is that the traffic on this 
newsgroup have increased recently.

Duncan, thanks for the details on the reason why pan is using so much memory. I 
wonder if it is possible to reduce the number of headers kept in memory in the 
ways I suggested, i.e.

1) Do not read in all headers, but only those that are specified via filters 
(e.g. to certain authors/titles). I am already filtering the headers I do see 
already, as there is no way I can deal with all the millions of headers, so I 
only ever see a very small subset of these anyway. I know that if a header is 
not read in, I will not be able to see the associated article, but in order to 
reduce the memory usage, it seems the first thing to give up is these filtered 
headers that I would most likely see even if they are read in anyway. Can this 
be done, or is there some reason that all headers have to be read in?

2) Read in N day's worth of headers, but not necessarily the last N days, i.e. 
read in older headers, starting from an earlier date than today, e.g. 10 days 
of headers starting 10 days ago (i.e. headers from 10-20 days ago).

These seem to be simpler to implement than changing the way pan handles headers 
(using database as you suggested). I am not familiar with programming 
newsreaders, so perhaps I missed something?

I am using pan for Windows because pan is the only newsreader I found that 
could handle newsgroup with large number of articles, I have an Intel Mac as 
well as the Windows 7 laptop that I can read newsgroups. Actually I would 
prefer to read news on my Mac (which has 16G of real memory), but I haven't 
found a newsreader that can read in more than 100K of headers. Is there any 
alternatives to pan that can handle huge number of headers? I am actually 
trying to see if I can run pan on my Mac now, but I can't find any pre-built 
binaries, so I am trying to compile it now, but this seems non-trivial....

Thanks again,

Kish Shen




________________________________
From: Steve Davies <address@hidden>
To: K Shen <address@hidden>; address@hidden 
Sent: Friday, 19 October 2012, 14:24
Subject: Re: [Pan-users] pan for Windows crashes when reading large newsgroup


On 19 October 2012 07:08, K Shen <address@hidden> wrote:
[snip]
>
> I have been using a 32 bit x86 Windows XP laptop with 2G of real memory up
> to 3-4 months ago, which was replaced by a 64 bit x86-64 Windows 7 laptop
> with 4G of real memory. This was about 1-2 months before I noticed the crash
> problem, and I don't know if this new configuration is important for the
> crashes (I have not seen the crashes on the old laptop).
>
[snip]

Just a very quick data-point... The Pan installer installs a 32-bit
binary. This is the only build environment that I have, so that's all
there is. I am not even sure if all of the relevant libraries can be
built 64 bit under the mingw environment. A 32 bit app will wun under
a 64-bit operating system, but gains none of the benefits.

As a result, Pan is limited to the memory addressable by a 32-bit
application, which is 2Gb under windows. (32 bits allows 4Gb to be
addressed, but 32 bit apps only get 2Gb for a number of reasons which
I would not explain well)

Hope that helps,
Steve