Re: [Nmh-workers] new command lacks lock

[Eric Gillespie]

On Mon, Oct 14, 2013 at 3:26 PM, Ken Hornstein <address@hidden> wrote:
>>Even on local disk, I expect it makes a noticeable difference. Right
>>now it takes 0.009s to run, with 940,047 files in 282 directories,
>>11,994 unread total.
>
> Ok, but it sounds like you're not scanning all of those directories, right?

Sorry, I don't understand the question. As currently implemented, new
does not scan these directories, it only opens sequence files. I
thought you were proposing changing that.

> I am not interested in having a feature regression, but I am wondering
> how bad this really is nowadays.  How long does it take to do a "ls" on
> all of those folders?  (Assuming you're not running a "ls" that requires
> a stat() on each file, of course).  I use AFS over the Internet, and

Hah! Before I changed it (and pulled the code out to crawl_folders.c),
folder.c *was* stat-ing every entry it read! It still does so on
systems that don't have BSD's d_type innovation (Linux does).

> some quick tests suggests to me that even in that case readdir() is
> pretty quick.  If it's still lousy for you, then I'm willing to just

As I said, I don't use mh on NFS anymore, and maybe we don't care to
support giant folders on network storage with something as esoteric as
this (has anyone other than me ever run this before last week? :)

I tried a little test of creating a 140K file directory and ls-ing
that (yes, I verified with a separate strace ls run that it's not
stat-ing them). Consistently over .5s. Times 280 folders, that's
ludicrously slow.

But not all my folders are that big; we're probably only talking about
a few seconds, which is not too bad if you're stuck on NFS.

On local disk, it's only crazy slow: consistently > 0.16s, so new on
hundreds of giant folders would take tens of seconds. Maybe only a
second or two with my actual mix. I could probably put up with that.

It offends me aesthetically though: that's a ton of wasted work.
Reading the directories is completely unnecessary.

Anyway, I've never been more than a drive-by contributor, so I
certainly won't put up a fight if you decide to change it.

Thanks!