First of all I need to state that I'm just an undergraduate student
and don't speak for FreeType.
Running continuous integration on Savannah isn't really an option.
The regression server can poll Savannah every 10 minutes. IMHO it is
good enough for a project that makes one commit a day on average.
According to their docs, Savannah can send notifications about commits.
The user can also start regression manually. Some commits don't need
regression testing.
> However it is not really practical for me to set up servers for your organization.
But a detailed installation instruction would be nice to have.
> "Improving the UI" is really vague
It is convenient to have a dashboard that allows to compare any pair of
revisions, watch the job queue, cancel and reorder jobs in the queue,
and manage disk space.
I'd also like an option to test a patch before it is committed. Doing it locally
is not practical because few developers have access to all supported
platforms and architectures. For instance, I don't have a big endian box
or a box that enforces aligned access to memory.
> ... switching out the comparison tool used in the scripts shouldn't be difficult.
Of course it is not, but bmpcmp clips out unchanged part of the image and
generates several layers for mouse-over action. So some changes
are required.
I'd also like to see more error checking in the scripts because every external
program can fail for a variety of reasons.
Direct diff of ftbench output does not make sense because small variations
in the run time will create many false positives. It needs a custom processor.
Most likely FreeType already has a collection of fonts for testing, but I
cannot find any references. A lot of fonts, including some broken ones,
can be extracted from PDF files. An extraction tool would be quite handy
to have.
Besides raster image comparison, there's Valgrind, Memendo, and Coverity
that will surely find something to do for developers.