[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Nmh-workers] MH-W intro/help request

From: Erich Boleyn
Subject: [Nmh-workers] MH-W intro/help request
Date: Mon, 01 Dec 2014 12:54:48 -0800

[NOTE: I apologize in advance if this seems long-winded and if it covers
 anything that has already been hashed on the list, but what seems to me
 reasonable searches have not revealed anything, so here it goes...]

Greetings all,

I have been working on a project I tentatively call "MH-W".  I'm very
busy/have little time, but I finally got tired of not having a Web
interface to my email, even though I have nearly no experience in
web development per se.

MH-W is a C++ program using the FastCGI interface to a web server
which, as much as possible, performs only the "Web Services" part
of email.  I picked FastCGI because it is very easy to connect
to J-Random-Webserver, usually just a few config file entries and
off you go, no recompiling the server or other nonsense required.
For example, it also currently has no other non-UNIX-basic library
requirements than the basic FastCGI, so it is easy to compile.

It tries to make as few policy decisions or extra steps as possible
and otherwise rely upon the MH programs to do all the work.  The
result is that you (mostly) get to interact with it the same way
you would as your command-line.  That is:  You can still be
interacting with MH in other windows and get the right intuitive
results.  You can display a message in the web interface, and just
like the normal default in MH, it will mark the "current message"
properly, refreshing a "scan" window re-runs the scan, etc.  As
much as possible it uses MH programs for *everything* it knows about
and all interactions with your email (with a small caveat, see below).

Features/Basic Structure it has today:

  --  Simple Session Management:  Cookies are generated using /dev/urandom
        and expire within a reasonable period of time, cookies are NEVER
        saved to a file.  (I considered client certs, but it is a bit
        of a quagmire and I could figure out no way to make the webserver
        interface nearly as independent/clean as I have with Cookies)
        When you click on "Logout User ..." it will completely forget
        the Cookie/Session.

  --  Simple Site/Page Organization:  Like MH, you fundamentally have
        a "Folders View", a "Scan View", a "Message View", and a
        "Component View".  Some can be adjusted/configured.

  --  "Scan View" overview of important features:
        --  "Pick" list has very simple syntax passed right to MH pick.
        --  Different colorization of lines which are in the "unseen"
                list vs. "seen".
        --  Checkboxes down left side for selecting messages, a handful
                of accelerators for selecting message groups (all on/off,
                select/deselect all above/below X place on the page), and
                a set of convienient buttons such as "DeletePack" and
                "RefilePack" to perform what should be the obvious actions
                of "Delete" or "Refile" of the selections then packing
                the folder.

  --  "Message View" overview of important features:
        --  Header processed to show you the important fields.
        --  Versions of the "Message View" available:
                --  HTML view (full/normal HTML passed through).
                --  Sanitized HTML view (all Javascript removed, no panes).
                --  Locked (locked-down) view (Sanitized no images).
                --  Raw view (de-tags and shows whole message as text).
        --  Default view is "Sanitized" or "Locked" as configured.

  --  "Component View" overview of important features:
        --  Shows you the list of components.
        --  You can attempt to view in the browser or extract any component
                to a file.

NOTE:  It is really not intended to be an email reader for completely
insecure users and protect your system from them by assuming they might
do Bad Things.  For example: It currently gives you pretty full "pick"
capabilities directly from the URL, and I think that is probably OK,
with the model being that this is a secure way to log in and read/
interact with your email, but make no mistake that you are logged into
the system.

Attached is an example of the "Scan View" and an individual message
displayed in the "Message View", both of this email list, to give
people an idea of what it looks like in practice right now.  The
checkboxes were selected via one click on the "v" button to the left
of the top one checked (the "select/deselect all below this place"
feature in use).

It is *not* yet ready to distribute, as there are several major things
missing (or I am in the process of overhauling):

  --  Real Authentication
        I have the internal structure for it (cookies as mentioned)
        but have not yet figured out how to make it call the proper
        PAM mechanism to do it.  Maybe if someone has a good pointer
        for how to properly call out to PAM, that would help, I have
        had no luck in finding one in my Copious Spare Time (of which
        I have none).

  --  Real "Compose" capability
        I still don't have even the basic one working, and I want the
        attachment MIME support to be in it too.  I admit that a few
        pointers to suggest something good here might be good too given
        I have literally no exerience in how to do non-trivial editing
        in a window.  (For example, just a big text box is the naive
        idea, but I bet there are better I just have no idea what the
        "right" ones are...)

  --  Markup pages for Layout
        The Layout/Formatting of the "views" are currently fixed,
        and while they work well, I'm sure the hackers out there would
        Complain Bitterly about such-and-such spacing or whatever, and
        I'm Annoyed myself that I have to recompile each time I tweak
        the layout.  My current plan is to (very soon) replace the
        layouts entirely with a mechanism much like MH's own format
        files for replies, scans, etc.  with building blocks using
        the mechanisms I've come up with.  I.e. it would automatically
        insert the right stuff for the checkboxes on each scan line
        and macro-ify my currently hardcoded choices for what do to
        with selection lists like "DeletePack", "Delete", "RefilePack",
        etc.  This kind of thing could possibly allow insecure users
        as well...  we'd just create pages that only some authenticated
        users could use and disallow the flexible scan controls or
        whatever to the "insecure" users...  but I consider this to
        be Future stuff that is not for worrying about right now.

  --  HTML Sanitization (to remove Javascript or other active web stuff)
        I have a basic HTML Sanitization mechanism using the somewhat
        older program "html2text" (which doesn't work too well), and
        am contemplating the best next step.  I have had recommended
        to me using either "tagsoup" or Google's "streamhtmlparser",
        both of which would allow true parsing/traversal of modern
        HTML, to strip out the active elements but then re-emit the
        passive ones.  As before I'm open to suggestions.

The first 2 are certainly critical before any kind of distribution,
probably the 3rd as well.  Maybe if a few people who know about the
areas I lack and/or are motivated wanted to contribute I could be

So, if I'm not ready to distribute it, Why am I writing to the list now?

The move to NMH 1.6 added some features I needed (most important was
being able to extract a specified component to standard out, yay!)
and was a great step ahead, but I still need at least 1 more feature,
and 1 more to be able to claim "absolutely no peeking into the
internals of MH or weird workarounds".

Big request (the one I have only a poor workaround for):
   I need a feature in the MIME support (or just help if I'm totally
   confused).  I have for the life of me not been able to figure out
   how to get "Content-IDs" listed.  We really need those to be able
   to display inline images in email.  The "mhlist" program will list
   the file names or if something is marked as "inline", but not the
   content-id itself, which is how the HTML in the email identifies
   which picture or other thing is inlined in the proper location.
   I have a fallback right now which just finds the Nth inlined
   item and uses that, but that is a poor substitute and I have seen
   it fail.  Ideally, it would be useful to have both "mhlist" support
   listing Content-IDs and "mhstore" to be able to use them to identify
   the part in question, but even if just "mhlist" listed them, I could
   perform the extra step of listing, getting the "part", then using
   that with "mhstore".

Lesser request (but still good from a cleanliness perspective):
   Right now, there is no way I have found to get which messages are
   "unseen" (or some other mechanism) other than effectively listing
   all the unseen messages in a folder.  The 2 methods I know of,
   both using "pick" and using "mark" list them all.  If you have
   a large email folder this is prohibitive.  So, either pick or
   mark having the capability to use the sequences to "qualify"
   an existing message list would be good.  My workaround right now
   is that I at least use "mark -list -sequence <foo>" to get the
   sequence, but yes then I have to parse the whole sequence list
   to figure out where my current pick list resides.

Anyway, thanks for everyone's time, and hope I can get through the
"required" bits to get this out there to all interested in the not
too unreasonably far future...  ;-)

    Erich Stefan Boleyn     <address@hidden>     http://www.uruk.org/
"Reality is truly stranger than fiction; Probably why fiction is so popular"

Attachment: mhw_screenshot_scan_show.png
Description: PNG image

reply via email to

[Prev in Thread] Current Thread [Next in Thread]