bug#36714: 27.0.50; Gnus nnmaildir taking long time to recursively visit sub-Maildirs

[Eric Abrahamsen]

On 07/22/19 10:44 AM, Jean Louis wrote:
> * Eric Abrahamsen <eric@ericabrahamsen.net> [2019-07-19 19:24]:
>> > Thank you for helping. Yet it is not usable for
>> > me. I have too many maildirs and I was thinking
>> > Gnus would read it as maildirs, instead it started
>> > creating .nnmaildir directories inside with copies
>> > of those emails for its own way of processing.
>> 
>> Yeah, I don't think there's any way around the creation of the
>> .nnmaildir directories, at least not given the way Gnus currently
>> functions. They aren't actually copies of the emails -- just vectors of
>> headers for each mail -- but obviously if you've got a lot of mail they
>> still take up a lot of space.
>
> If they are not copies, they took as much space as
> original emails.
>
> That is opposite and contradictory to what Maildir
> is supposed to be.
>
> nnmaildir is thus not described well enough in the
> Gnus manual, it is bug in itself.
>
> When somebody mentions "Maildir" that means
> managing emails in Maildir folders, and not making
> indexes or vectors inside of those Maildir
> folders, taking up gigabytes of spaces and
> basically endangering integrity of files of the
> user. 
>
> Thus that behavior of nnmaildir is bug.

Gnus operates as a general newsreader/email client, not a dedicated
maildir reader. The maildir functionality needs to fit in with its
existing paradigms, and right now that means maintaining nov header
files alongside the messages themselves. Otherwise actually listing
groups and reading messages would be unbearably slow. I agree it's
essentially a bug, but it's one that can't be fixed without some
fundamental alteration of how Gnus works.

I also don't think this threatens the integrity of the user's files: the
extra data is maintained in parallel, and doesn't interfere with the
messages themselves.

> One shall explain it very well in Gnus manual that
> nnmaildir is not managing Maildir folders but
> rather using Maildir folders to make indexes,
> vectors, having some kind of news overlay on top
> of Maildir folders.
>
> And one shall mention the limit, as it simple does
> not work over certain number of Maildirs. I do not
> know how many, as I do have many maildirs. It is
> unusable for me.

The limit (defined as "the point at which the user gets annoyed") is
going to be different for different machines, and different users. But I
agree the manual should provide a more prominent warning.