Re: Jamming up with mutex_lock

[Andrew Daviel]

Thanks for the reply; I'm in a bit over my head, usually I stick to just 
building packages and writing Perl scripts

On Tue, 19 Jun 2007, Joe Maimon wrote:

> The other threads continue to operate.
>
> You then probably run into a ulimit condition.

AFAIK there is no ulimit set, unless it's hidden somewhere.
I'm not sure there is any real connection between the occasional
read error (maybe spamassassin was slow, or the CPU was busy) and
the real problem - lockups

> > When I look at the processes, I see two or more copies of spamass-milter
> > in sleep (S) state as well as the parent in sleep (Ss1) state.
>
> Are you displaying all the threads?

No. That was with "ps ax".
Right now, it is not jammed, but I see one child process in mutex_lock.
The parent process has some 20 threads with various start times going 
back 3 hours or so.

I presume the threads are supposed to exit when the message they are 
handling is done; clearly some of them aren't. Some of the mail we get is
presumably from ratware that may not obey SMTP, like just dropping 
connections without sending QUIT, or sending DATA without waiting for an 
OK from RCPT TO. I'm not sure how that impacts milters.

> If you suspect the milter calls unsafe functions, surround them with mutex's.

Have to read up on that... I think it's a red herring, though; localtime
is only called if the sendmail "b" macro isn't passed and the strerror
calls are mostly precursors to dying.
> Try running this on a recent Debian instead.

Not feasible. RHEL (actually Scientific Linux) is our site standard, 
and while I could try something different on a non-production system it 
would doubtless work just fine because it wouldn't get the traffic.
I might be able to try a newer sendmail.

> Perhaps you could show the patch?

http://andrew.triumf.ca/email/spamass-milter.patch.jun18

which is now a mix of things we need (match_gecos), things I've told the 
users they can expect (reject_thrsh, custom rejection message), things
I thought were a good idea (msg_size), and things I'm just grasping at 
straws with (mylower, strerror_r). strerror_r is a pain because there's a
Posix one and a Gnu one with different behaviour, and a different 
default depending on which glibc version. The Posix one is threadsafe.

(while the -x option would work instead of adding match_gecos, it also
results in all .forward files being expanded, which can cause a local 
user's preferences to be applied to a forwarded user with the same ID,
e.g. mail sent to  "jill" which is forwarded to address@hidden
gets mary's score and whitelists.)

--
Andrew Daviel, TRIUMF, Canada
Tel. +1 (604) 222-7376  (Pacific Time)
Network Security Manager