[Pan-users] Re: One group suddenly ignored

[Duncan]

Glen Walpert posted on Sat, 09 Apr 2011 09:55:24 -0400 as excerpted:

> OK,  I will compile and install 0.134 as soon as I can finish some other
> work, and let you know the results.

It's narrowly possible it's something else fixed in the new version too.  
Not likely, but at least with it installed we're all pretty much on the 
same page.

> Testing with 0.133 now:
> Get all headers retrieves a lot of headers, but does not get all the
> missing headers, and they are all marked as read!  Even stranger, if I
> wait a while and get new headers (without deselecting
> sci.electronics.design), some new headers are retrieved, but they only
> show up in the headers pane when "show only unread messages" is
> selected!

As I said, I don't know how that data fits the whole picture yet, but it's 
good to have.

> The fresh server config (I even uninstalled pan, deleted the .pan2
> directory, and reinstalled with a new manual config) does not affect the
> problem.

OK, can't be a simple server renumbering or the like.  One possibility 
ruled out.

>> The patch alluded to above switched pan to using a 64-bit number for
>> the same thing.  That's not going to run out in the foreseeable future!
>>  But that patch didn't make it into 0.133, only 0.134.  Thus, the fix
>> in this case is to upgrade to 0.134.  However, a sci.* group doesn't
>> normally allow binaries (does it?) and as such, is quite unlikely to
>> have surpassed the 32-bit limit of some 4 billion plus posts, in which
>> case this patch won't solve the problem you are seeing.
>>   
> There are less than 200,000 posts to sci.electronics.design on astraweb.

FWIW, the per-group sequential would include all posts the server has ever 
seen for that group, possibly including some it didn't actually post 
(depending on whether it numbers before it spam-filters, etc).  So this 
one's still a narrow possibility, but very narrow, as 4 billion posts plus 
is a LOT of posts for a primarily text group such as the sci.* hierarchy 
will be.

>> Third... not so good news.  There has been a longstanding but quite
>> obscure bug in pan for quite some time, related in some way to cross-
>> posts.  The bug is obscure enough not to affect most folks, and I'd be
>> unsure it was even there, but for the fact that I have it myself!
>> Unfortunately, it's obscure enough and complicated enough nobody has
>> been able to trace down the problem, and it remains unfixed.
>>
>> What I know about the problem so far is that it is somehow triggered by
>> crossposts to multiple groups.  I /think/ it only happens when one
>> group involved in the cross-post gets ahead of another, perhaps you
>> don't visit or update the behind group for some days, or perhaps you're
>> just adding it as a new subscription after seeing sufficient cross-post
>> activity to become interested in the other group (my case here). 
>> Somehow, something about the cross-posts gets pan mixed up, with the
>> result being, apparently, a case of the first possibility above -- pan
>> somehow records way high message sequence numbers for the one group,
>> perhaps recording the numbers for the first group cross-posted to, to
>> the second instead of using the lower numbers of the second group.  As
>> such, it then believes any new messages it sees are in fact years old
>> and refuses to download them!
>>
>> If you are seeing this rather obscure behavior, I'm sad for you, but
>> glad, in that we now have another data-point to compare against mine,
>> and maybe some progress can finally be made on this thing!
>>
>> Two questions should help prove this the case or not:
>>
>> 1) Does the problem group get cross-posts from another group you
>> regularly visit, or may have visited in the past?  (If no such
>> cross-posts are involved, it's unlikely you're experiencing this same
>> bug.)
>>   
> Some users of sci.electronics.design do inexplicably routinely crosspost
> text only messages to alt.binaries.schematics.electronics, another group
> I follow, so this may be the crosspost related bug.

Ugh.  Sounds like you may have the bug I have.

>> 2) If you start a clean pan instance (say by moving the ~/.pan2
>> directory elsewhere, when pan's not running of course, so it has to
>> start a new one), setup the server, and ONLY visit the problem group,
>> WITHOUT visiting anything else first, so the ONLY data it has is from
>> that problem group, the group should load just fine.  However, start
>> trying to bring the saved data from the other groups back in and the
>> problem group may go unresponsive again.  The existing posts will
>> remain (until they expire anyway), but pan will refuse to update the
>> group any further.  (This is what I see here.  Again, if with a
>> completely clean pan instance, with the problem group the FIRST you
>> visit so no history or possibility at all for the cross-posted messages
>> to interfere since the other group hasn't been visited yet, if it does
>> NOT load just fine, you're seeing a different problem.)
>>   
> I got exactly these results, but strangely the problem re-occurs when I
> load other groups not including the cross-post suspect
> alt.binaries.schematics.electronics.  I am not sure if the problem would
> re-occur in a few uses even with no other groups visited yet since I
> added new groups as soon as it appeared that sci.electronics.design was
> working, I will test that later, after taking care of some other work.

FWIW, there /is/ a somewhat nasty and inconvenient work-around... as long 
as the group stays stable by itself.

When pan first starts, it looks at the PAN_HOME environmental variable, if 
it's set in pan's environment, to see where it should look for and/or 
place if it's not there yet, pan's data dir.  The default, if the variable 
isn't set, is ~/.pan2.

I use this to setup multiple pan instances, here, binary, text, test.  
Each has its own starter script that points pan at its own separate data 
dir.  For files such as the scorefile that I want common to all instances, 
I use symlinks.

It works quite well for that, but could also, if it came to it, be used as 
a workaround for this issue.  Setup a new pan instance that you use ONLY 
for the problem group.  Nasty as I said, and inconvenient, but it should 
work.

Meanwhile, if you are indeed affected by the same bug I am, we have at 
least one important new data point.  I had only seen it on the gmane.org 
list2news newsserver, which is... somewhat peculiar in its setup since 
it's gating lists, not directly peering news.  If you're seeing this bug 
on astraweb as well, we now have at least one more known fact on it -- it 
is **NOT** gmane specific!  That in itself is a decent advance, since 
while there have been a couple people who reported something I /thought/ 
looked like the bug, this is the first one that's looking to be so close 
to a match, and being possibly the only one I knew affected, and only on 
gmane, I was beginning to think it might be some gmane peculiarity.  Now, 
it seems not.

But upgrade to 0.134 and see if the problem goes away.  For you, I hope it 
does, as this bug's a bugger for sure!  But if it's the same bug, just 
having your second datapoint has already given me more info on the bug 
than I had before, since as I said, I was beginning to think it must be 
gmane-specific.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman