Re: [Pan-devel] Re: Ya know Pan2 is really single threaded, right?

[Calin A. Culianu]

On Sat, 17 Mar 2007, Duncan wrote:

> > Is this intentional?
>
> It's known and intentional, yes.  Older pan (pre 0.9x, coded in C instead
> of C++) was indeed multithreaded in its downloading.

Ah, ok.  I figured.  The code is otherwise pretty tight and such an 
'oversight' was likely intentional. Thanks for answering that question!

> Meanwhile, I've found it /much/ more my style and much less likely to
> stall ongoing downloads if in the binary groups I download everything to
> cache, then do local processing as a second round, after everything's
> already downloaded.  Of course, pan's default 10 MB cache isn't large
> enough for this, but that's easily solved by tweaking the setting in
> preferences.xml.  (As with several "advanced" settings, there's no way to
> tweak it in the GUI prefs, only by editing the config files directly.)

Hmm.  Ah yes that didn't occur to me but I can see how it works and why 
it works.

Still, you have to admit it is a bit labor intensive from the user 
(downloading lots of binaries is a 2-step process in the UI) and is also 
wasteful of diskspace.  For example: if you are processing a huge NZB 
queue (downloading a DVD iso can be like 4GB!!).  If it's posted as 100 
50MB .rars, you waste all 100*50MB in the cache rather than just 1*50MB 
for each attachment.

> As I said, however, I've never seen Charles explain his choice, but
> expect I'd learn something about threading as well as about pan seeing
> it, so thanks for asking and hope he replies too. =8^)

I have been reading the code and trying to add a worker thread pool model 
to the code.

I can see now that it introduces a lot of complexity and potential bugs to 
the code -- already the code assumes that everything runs in 1 thread.

But I will do my best to aviod race conditions by locking shared data as 
best I can, and to only introduce threading where it will actually benefit 
the user (better network utilization, not stalling on uudecode, etc).

I have experience with multithreaded coded so hopefully I can do this 
somewhat elegantly and in a bug-free manner.

Likely he didn't make the code multithreaded because his initial pass was 
to just get Pan2 working, and perhaps he intended to go in later and added 
multithreading?

Anyway I am taking a stab at this now -- have been spending a few hours 
on it already.  I'll let you guys know if I give up, lose interest, have 
or actually have an implementation that works.

Then you guys can decide to keep it or not.

Thanks for the info and yes, I also would like to hear Charles' thoughts 
on this.

-Calin