[Monotone-devel] parallel-tests merged

[Zack Weinberg]

I have just merged the .parallel-tests branch to the mainline.  This
means you can now run "make -jN check" and get some pretty hefty
speedups.  By way of illustration - on my older, not very fast
anymore, hyperthreaded i386 box, make -j1 check takes 20 minutes, -j2
takes 15, -j3 is a little faster than that.  On my rather newer
quad-core amd64, make check goes from 10 minutes at -j1 to 3:30 at
-j4.  Full statistics are attached.

I only implemented parallelism among individual testcases for Unix;
Windows builds can still get some benefit by being able to run the
unit tests in parallel with the integration tests (i.e. -j2 will do
something useful, higher numbers won't).

There are a few visible side effects: most important is that
"unit_tests" is now called "unit_tester" and does nothing useful if
you invoke it directly.  Run the unit tests by hand with
"./run_unit_tests".  Also, there are now intermediate per-testsuite
directories between tester_dir and the per-test directories, and the
progress scroll now mentions each test only when it is completed (this
is unfortunate but unavoidable).

The code for all this, sadly, is a bit of a tangle: Lua calls back
into C which forks and invokes Lua again in the child, plus lots of
fun with file descriptors.  Improvements are welcomed.  Note that in
the original, I was not bending over backward to avoid use of
stdin/out/err in the child processes; in fact, I made the per-test
logfile be stdout.  Unfortunately, this led to a bizarre bug where
text that should have been written to stdout showed up in some other
file altogether - but only intermittently and never under the
debugger.  It was easier to make sure the child never touched stdout
than to figure out what was really going on.

(In case anyone is wondering why we don't use the same code path on
Unix as we do on Windows - where, by necessity, we restart the test
runner in a special mode in each child - it's because -j2 check on the
hyperthreaded i386 was about two minutes faster with bare fork().)

zw

parallel-tests-perf.txt
Description: Text document