[Pan-users] Re: Download stuck

[Duncan]

António Godinho <address@hidden> posted
address@hidden, excerpted below, on  Wed, 08 Nov
2006 14:01:54 +0000:

> On the new versions after some time the program stops
> downloading and keeps the remaining files to download on the list but
> without retrying. It’s the a known problem to be fixed or just happends to
> me?

It's a known problem, or rather at least two problems.  One of them was
supposed to be fixed with 0.118, I believe.  However, another one is a bit
tougher and not entirely pan's problem, tho pan could possibly handle it
better (but would require changes deeper than will go in before 1.0).
There's a known workaround tho.

The problem is this.  One of the features of yenc is that it includes crc
checksumming features designed to prevent corruption in transit and
propagation.  If a particular attachment doesn't match the crc and/or
size, pan knows that it's corrupt and won't save the attachment.  However,
it doesn't know to try again or how to use par2 for repairs or the like. 
It just sits there with the bad file, doing nothing.  (Old-pan I believe
just silently saved the corrupt file as if nothing was wrong.  It might
have put something in the log, but folks wouldn't always catch it if it did.)

The workaround happens to be the way I use pan anyway.  Rather than save
attachments directly as they are downloaded, I use the download to cache
function, downloading a whole bunch of stuff (often multiple gigabytes at
a time) without having pan do anything more with it right then.  Since pan
isn't trying to save the attachments, it doesn't know or care whether the
attachment is corrupt or not -- it downloads what it has and goes on to
the next one.

Then I go back thru after everything is downloaded and I can work with it
without waiting for the download, and decide what to do with everything. 
If it's not worth saving, I'll delete it.  If it's worth saving, I'm still
working with the posts and therefore have all the extra metadata such as
poster, posting date, and subject line, still available.

Since it's all local when I'm working with it, if something fails to save
then, I know right away and use the save post text (not attachment)
option, saving all the pieces I'm working with to a scratch directory.  I
can then use uudeview to decode it.  uudeview doesn't care about the
corruption, but the resulting corrupted files are still useful,
particularly when the posting was accompanied by par2 repair files,
because the damage will often be limited to a single block or two, and is
thus easily repairable.  Repair works much better having downloaded the
corrupt file rather than skip it, because only a block or two of the (say)
10 blocks are damaged, meaning 8 or 9 are fine.

The caveat with the download to cache method, however, is that you have to
set pan to use a larger cache, since it's only 10MB by default.  That's
enough to cache several text sessions, and it works fine for direct-save
downloads, but try downloading a few gigs of binaries to cache and you WILL
be frustrated, unless you adjust the size of cache first.  That's not a
GUI config option, but you can set it by directly text-editing
preferences.xml.  I have my binary pan instance cache symlinked to a
dedicated 12 gig partition, and have the cache size set accordingly.  That
works quite well. =8^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman