Hi all,
We're having some seemingly monit-related misbehavior on one of our machines.
We use monit to control a pool of mongrels, 15 per host.
Our pool runs on ports 8000 through 8014. Of these mongrels, most of them work normally.
However... occasionally the mongrel on port 8014 of a given host will just act *crazy*. Either
it won't start up after a monit restart is issued to the pool, or it will fail to restart when it is told to do so.
Here's what our monit config for port 8014 looks like:
check process mongrel_8014 with pidfile /home/rails/application/production/current/log/mongrel.8014.pid
group mongrel
start program = "/usr/local/bin/mongrel_wrapper start 8014"
stop program = "/usr/local/bin/mongrel_wrapper stop 8014"
if failed url
http://127.0.0.1:8014/status and content == "OK"
with timeout 120 seconds for 2 cycles
then restart
if memory > 120 Mb then restart
This fragment is included by monitrc - in our local environment (Ubuntu Hardy on amazon ec2), I'm tending to drop these rules into files in /etc/monit.d.
This is monit 4.10 - hand-compiled, not installed from the ubuntu .deb.
Has anyone seen an issue like this previously? We'd be happy to pay someone with the appropriate skills to debug it - drop me a line if you're interested in that work (and qualified, of course).
--elijah