Samizdat benchmarks: mod_ruby vs mod_fastcgi vs mod_fcgid under Apache2 mpm-worker and mpm-prefork

[Dmitry Borodaenko]

Now that Samizdat works again with both mod_ruby and FastCGI, I've run
some benchmarks to see which of these modules is more preferrable. As
I've already noted in install.txt, mod_ruby turned out to be slightly
better than the others. Although dispersion of test results in most
cases is bigger than difference between modules, there some things
that stick out:

1) mod_fastcgi makes Samizdat leak a lot of memory, there's no such
problem with mod_ruby and mod_fcgid (I know that mod_ruby doesn't like
mpm-worker so I didn't even try it, would probably be even worse).

2) For some reason, focus update is several times slower with
mod_fcgid/mpm-prefork than with all other combinations.

3) While having a comparable speed in all tests, mod_ruby seems to use
less memory than mod_fcgid, not to mention mod_fastcgi.

4) Even in the worst conditions (with cache being effectively negated
by parallel run of focus updates, imitating a real-time discussion),
Samizdat is able to generate at least 20 front pages per minute (50
with mod_ruby) on my 1.5 GHz laptop. In normal conditions (with more
hits than updates), it should be able to handle around 200 thousand
hits per day on the same machine.

All stats (mean hits per second, maximum size of a single Ruby and/or
Apache process):

libapache2-mod-fastcgi/apache2-mpm-worker
item.rb (1)             20.32   14.48   11.65
index.rb (2)            2.63    2.45    2.10
index.rb +              0.49    0.39    0.34
resource.rb (3)         17.15   11.88   11.27
memory: 25*ruby (95m max), 2*apache (7m max)

libapache2-mod-fcgid/apache2-mpm-worker
item.rb (1)             17.93   13.37   9.95
index.rb (2)            1.68    1.58    1.81
index.rb +              1.19    0.98    0.71
resource.rb (3)         8.67    8.98    10.65
memory: 50*ruby (29m max), 2*apache (7m max)

libapache2-mod-fastcgi/apache2-mpm-prefork
item.rb (1)             19.65   14.06   17.92
index.rb (2)            2.43    2.20    2.15
index.rb +              1.22    0.53    0.50
resource.rb (3)         1.25    14.08   9.64
memory: 21*ruby (82m max), 22*apache (4m max)

libapache2-mod-fcgid/apache2-mpm-prefork
item.rb (1)             19.51   18.55   18.53
index.rb (2)            2.48    1.88    1.46
index.rb +              1.51    1.56    1.51
resource.rb (3)         0.32    1.16    2.03
memory: 42*ruby (29m max), 21*apache (4m max)

libapache2-mod-ruby/apache2-mpm-prefork
item.rb (1)             23.02   20.85   23.36
index.rb (2)            2.00    2.51    2.56
index.rb +              1.01    0.78    0.93
resource.rb (3)         7.08    6.84    6.23
memory: 20*apache (29m max)

---
1) minimal page, no database access
 /usr/sbin/ab -qdS -c20 -t60 http://imc-by/item.rb

2) front page, compressed, with session, cached
 /usr/sbin/ab -qdS -c20 -t60 -H'Accept-Encoding: gzip'
-C'samizdat_session=3d2a7cf4388ab70b6956df718e24a8f0' http://imc-by/

3) front page, with cache being flushed by parallel process
 /usr/sbin/ab -qdS -c10 -t60 -H'Accept-Encoding: gzip'
-C'samizdat_session=3d2a7cf4388ab70b6956df718e24a8f0' http://imc-by/
 /usr/sbin/ab -qdS -c10 -t60 -H'Accept-Encoding: gzip'
-C'samizdat_session=3d2a7cf4388ab70b6956df718e24a8f0'
'http://imc-by/resource.rb?id=5156&focus=323&rating=1&submit'

Versions:

apache2 2.0.55-4.2
libapache2-mod-fastcgi 2.4.2-7
libapache2-mod-fcgid 1.10-1
libapache2-mod-ruby 1.2.6-1

--
Dmitry Borodaenko