Re: [Pan-devel] My list of present problems with Pan.

[Duncan]

SciFi posted on Mon, 17 Oct 2011 20:13:48 +0000 as excerpted:

> I should probably post the concerns I'm still having with Pan.  I sent
> this to imhotep82 (Heinrich Müller, before he became judgefudge), and to
> lostcoder (K. Haley) some months ago.

Cool.

> My current Pan build can be seen with this post's headers under
> User-Agent (I'll send judgefudge the patches that "enhance" that
> string, soonish).

Very cool! =:^)

(I'm sure many of us still remember when the git-commit patch was added.  
We thought that great.  But the problem of so many git trees and branches 
that we need to identify them in the about box and user-agent string is a 
good problem to have, indeed! =:^)

> 1.
> 
> a)  For Pan's HTML highlighting feature (View->Body_Pane->Highlight
> URLs) we need the Tilde character '~' to be accepted as part of the URL
> (sometimes I call this a 'squiggle' <g>).  Presently, Pan is stopping
> the highlighting/copying at just-before this character when it's part of
> the honest-to-goodness URL.  All unix-type servers use this character to
> denote a user's "home folder" no matter how it is structured on that
> server.  We see these kinds of URLs all over the developers lists at
> Gmane and in the world-wide Usenet.  For now, we must do the drag-copy
> of the URL and manually paste it into our browsers.  There are several
> more characters that ought to be acceptable -- look at the way Gmane
> scrambles the URLs and email-addys, such Pan-highlighting does not seem
> to include '+' and '-', either; but don't take this as a sole-missing
> list.  ;)

> b)  Some other characters are being accepted by Pan's Highlighting code,
> which should *not* be accepted.  For example, when someone uses
> parentheses for the URL such as (http://whatever.com/) , the trailing
> right-parens ')' is being highlighted as if it is part of the URL, and
> being passed to the browser as well (which likely gets a 404).
> (I know this logic could make one go crazy trying to code-around it, but
> that's why we have open URL/URI type libraries, isn't it?  <g>)

The ) inclusion is the one that hits me most, here, as I follow gmane's 
g.c.freedesktop.xorg.drivers.ati list, which is mostly bugzilla mail 
forwards, and all the file attachments end up with the ) included in the 
URL. =:^(

+ in email addys is a needed addition too, as is ~ in URLs.  I guess I 
don't run into the - ones so often here.  Interesting.

Meanwhile, drag-copy, yes, but on a "decent" platform (kde with klipper 
and I'd guess gnome, I'm surprised OSX doesn't have something, tho 
perhaps they think it too complicated for the simple-minded users... come 
to think of it, that might mean gnome misses it too, but I KNOW kde has 
it! =:^), the selection/clipboard assistant should be programmable to 
recognize such things, and popup a list of actions as desired (with pre-
populated recognition and action lists based on native link-recognition, 
etc, but the option to expand or replace the pre-populated list as 
desired).

Thus, once I select the /corrected/ link, I get an automatic popup with 
actions including opening in various browsers (firefox and konqueror as 
graphic browsers, links and lynx as text-mode browsers, here), my text 
editors of choice (mcedit as primary and text-mode, alternative kwrite), 
image viewers (gwenview), plus various net-info and debugging choices 
(ping, traceroute, tcptraceroute, tracepath, mtr, host, nslookup, whois).

So there's no need to open/activate a browser and paste the link.  All I 
have to do is select the appropriate action from the klipper popup. =:^)

But while that does dramatically lower the frustration level, that very 
fact may have something to do with the fact that the problem still exists 
in pan.  The "itch" hasn't been bad enough to be worth the trouble for a 
coder, generally exactly the type to make best use of clipboard helpers 
and the like, to be worth scratching.  If they didn't exist, that feature 
may well work better in pan now than it actually does, as the existing 
feature is "good enough".  Oh, well...

> 2.
> 
> Every time I start-up Pan, the Header pane itself is initially "too
> wide" with its scroll-bar opening to the right-side.  In my setup, it's
> the Bytes column's numbers are barely visible, it's been scrolled-off
> the view that far.

No clue.  I don't have the problem here, but that's as likely to be due 
to my layout (full width header pane, all columns shown, horizontally 
maximized 1920 px wide pan window, 22" monitor with not too huge fonts) 
as anything else.

I do seem to recall similar issues some time ago, tho, but they're far 
enough back in the past that it's only a vague memory.

> 3.  Some of the IP-Number Displays still don't show the Port Number,
> mainly in the pop-up "yellow boxes", but in other areas also such as in
> the Posting Profile preferences at "Post Articles via:" drop-down list.
> My explanation:
> I use stunnel going to/from every NNTP server anymore.  All network
> paths in Pan need to be set to '127.0.0.1:<port>' for the corresponding
> entry in my stunnel.conf file.  I had been adding several lines in my
> /etc/hosts file to give various pseudo-hostnames to 127.0.0.1, just so I
> could get Pan to show each choice in a different way.  I don't know why,
> but OSX seems to want to go outside the box, still, to try finding the
> route for those pseudo-hosts, or some crazy related notion (blame Apple
> again for messing-up the standard *ix methods).  So I've gone back to
> entering precisely '127.0.0.1' inside Pan, but now I've lost being able
> to discern which line mean which path thru stunnel.  See?  ;)

Here's another possible workaround.  Whether it'll actually work, I'm not 
sure as I've never tried it, but as someone pointed out to me, all of the 
127/8 (class A) numberspace is reserved for localhost.  And I believe 
most at least *ix family IP-stacks shouldn't have problems with it, but 
it's possible some apps won't like it.

So try 127.0.0.1, 127.0.0.2, ... 127.1.0.1, ... 127.100.50.246, ... 
127.255.255.254.  (As you probably realize, 127.1.0.0 and 127.0.0.255, 
etc, should work as well, since it's a full class A, but eh...)

If it works as the theory says it should (you may need /etc/hosts entries 
for them), that should give you the identification you need even if the 
port is truncated.

Oh, and if you try it, please post your results! =:^)

> 4.  I would love to see some visual indicators for some text-formatting
> settings being used.  I need to see whether word-wrap is in effect, for
> the main issue here -- there are posts that don't much give a clue,
> because they would look "ugly" either way.  ;)  Things like that.  I
> would pick the area on the main Pan window to put these
> indicators/selectors, after the various "Match" icons and a
> separator-line thingy.

There's a *BIG* caveat here.  Pan already has a history of minimum width 
window issues, when used on netbooks, etc.  For similar display-size 
reasons, a dual-row toolbar (does gtk even do that?) isn't particularly 
viable, tho if necessary (and the toolbars are flexible enough to allow 
it), a full-app scrollbar (including the toolbars) can be added, making 
the vertical thing a bit less of an issue than the horizontal thing.

So if additional toolbar buttons are added, they *MUST* be either 
configurable to hide if necessary, or must do so automatically, thus not 
increasing the minimum window size.  And if they do, then some 
consideration should be taken as to button priority, so it's the least 
important buttons that get hidden, not the most important.

Of course the entire problem would disappear if only pan's toolbars were 
as configurable as the typical kde application's toolbars (tho there are 
exceptions; it's incredibly frustrating to see low-use button choices 
like help and about, when seriously useful action button choices simply 
don't appear at all!).  Hint, hint. =:^)

> 5.  Before accepting a "version 1.0" of Pan, I have written requests
> over the years to have more "multi-threading" incorporated into the
> program.

What's more distressing for me, here, is that particularly when one 
decides to cache decades worth of pan list history, etc, on cold-system-
cache as when first opening pan, it'll freeze not only itself, but the 
whole of X, for noticeable periods!  I can move the mouse, and anything 
already in memory seems to work, but all four spindle lights are on, on 
the RAID-1, indicating the system is accessing four separate things in 
parallel, and nothing responds until whatever it is is finished (still 
some time before pan actually shows up, but the RAID indicators go back 
to their more normal pattern, 1-4 blinking, not all four on solid).

AFAIK that's a kernel and hardware platform I/O thing to some degree, not 
entirely pan's fault by any means (in reality, the hardware and kernel 
shouldn't even allow that sort of hogging), but I believe it should be 
possible to program pan not to hit the system that hard.  I don't believe 
I've seen anything else do that since I went dual-dual-cores and 4-
spindle md/RAID-1.

> a)  For example, when I open a newsgroup known to be huge with binary
> files, everything is IMMEDIATELY STOPPED from going forth, _including_
> the download decoding queue which I thought was already a separate
> thread -- I can see such stoppage by visual observation of my modem's
> lights: they stop blinking while a newsgroup is being opened.  When the
> newsgroup is finally opened, the other functions continue from where
> it/they were stopped.

AFAIK, my experience above may shed some light on that.  Is the rest of 
the system interactive?  In particular, can you start a new app or open a 
new file that hasn't been accessed yet this boot (or since you did drop-
caches or the equivalent on OSX), so it's not in system-cache yet?  

If not, then it's I/O starvation of the entire system, not simply pan-
threading, altho as I said, it should be possible to code pan not to hit 
the system so hard, tho this is rather far from a solved issue in system 
I/O circles, for sure.

It's worth mentioning, BTW, that such lockups are rather more common on 
single-disk/single-core systems, and at least Linux itself has gotten 
better about it over the years.  It's significantly harder to do on a 
quad-disk Linux md/RAID-1 running on quad cores, than on a single-core 
single-disk system.  (It's also worth noting that this is a benefit of 
kernel RAID-1 as well, for read-only and read-mostly loads at least, as 
opposed even to RAID-0.  I was quite surprised at how well the kernel 
actually does in parallelizing access on the RAID-1 as opposed to RAID-6 
and even RAID-0.  System read-contention went down DRAMATICALLY, with a 
corresponding increase in responsiveness and speed.  But part of that was 
very likely due to the defrag effect of copying the existing system over 
to the new layout.  Honestly, I haven't done a mkfs and recopy from 
backup on the appropriate md/RAID since I downloaded the full gmane 
archive for several lists, including pan.user, and I really should, and 
see how much difference /that/ makes, before I complain too loudly.)

The point is, it's not necessarily entirely pan, at least from the data 
you've presented.  Part of it is likely disk/hardware I/O and kernel I/O 
scheduling issues.

Meanwhile, it's also worth noting that the rewrite to make pan disk-based 
rather than memory based would very likely solve this problem as part of 
the package.  That came up on -user again, recently, connected as usual 
with the switch-to-sqlite-for-the-backend proposal.

Thus, in reality, we're back where we were before the C++ rewrite.  The 
efficiencies of the new format did indeed buy pan some scalability in 
terms of memory, and a few years of calendar time, but we're back with 
the same problem, and the same proposed sqlite-database based disk-based 
solution, as opposed to the huge-in-memory tree that pan has used to this 
point, that we were discussing back before the rewrite.

I guess the good thing is that sqlite is rather better and more robust 
now, what with firefox's sqlite adoption and usage in the mean time, and 
the experience with the pan rewrite probably means pan will avoid certain 
implementation mistakes, as well.

But it still needs done...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman