[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Nmh-workers] Slowdown with a Populated Folder.

From: Ralph Corderoy
Subject: [Nmh-workers] Slowdown with a Populated Folder.
Date: Tue, 31 Dec 2013 18:54:13 +0000


I mentioned in the past finding my shell script that steps me through a
bunch of posts a bit slow on a large folder.  My inbox deteriorated
again and I had a bit of a look with `strace -f' to see where the time
was going.  It's mainly getdents(2) called by readdir(3).  getdents here
is called with a 32KiB buffer.  Here it is on a pruned and nippy +inbox.

    $ strace -e trace=getdents mark -s foo -list +inbox
    getdents(3, /* 1222 entries */, 32768)  = 32752
    getdents(3, /* 1206 entries */, 32768)  = 32768
    getdents(3, /* 1227 entries */, 32768)  = 32736
    getdents(3, /* 1214 entries */, 32768)  = 32752
    getdents(3, /* 1211 entries */, 32768)  = 32752
    getdents(3, /* 1202 entries */, 32768)  = 32736
    getdents(3, /* 415 entries */, 32768)   = 11240
    getdents(3, /* 0 entries */, 32768)     = 0

My rmmproc creates filenames like ,10139,1388316255.094920554 so as more
of those pile up, less entries fit in the 32KiB and more calls to it are
needed and there's more entries starting with `,' to be skipped over.
Clearly, I need to switch to ',' + base254-encoding.  ;-)

A shell-script that's kicking off a few nmh commands per user action
gets hit by this repeatedly and its adds up to noticeable lag.  Still,
I've confirmed the problem and know how to alleviate it.

A couple of points come from it...  Why does mark(1) above have to read
the folder when it's just doing a list of the .mh_sequences?  Is it
verifying the sequence's content against what still sits in the folder?
`mark -s foo -d all' gives an error if sequence foo doesn't exist, at
least with this old version.  Consequently, another mark is needed
first, the one above, to detect if the sequence exists.  A more useful
interface would cut the number of userspace commands that need to run,
and thus the number of getdents.

For reference:

    $ folder +inbox
    inbox+ has 6730 messages  (1-7210); cur=7190; (others).
    $ find `mhpath` -maxdepth 1 -printf x | wc -c

Happy New Year!

Cheers, Ralph.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]