bug#42137: cuirass: monitor evaluation status

[Mathieu Othacehe]

Hello,

With Hydra, it is possible for a derivation to have the status of all
"still succeeding jobs", "newly failing jobs", "newly succeeding
jobs". In a word, have a quick overview of an evaluation status.

You can have a look to
https://hydra.nixos.org/jobset/nixos/release-19.09-small where it's easy
to see that 1591716 caused two test failures, that were fixed right
after.

In Cuirass, it's only possible to know the status of triggered builds,
for each evaluation. For the last two months, the system installation
were almost all broken, I think that if we have had a page showing the
full status of system tests evaluation by evaluation, we could have
avoided this situation.

Also having the "newly failing jobs" list would allow to mail the people
that have commited changes between the last evaluation.

Of course this is easier said than done, it might imply to update our
database structure among other things.

WDYT?

Thanks,

Mathieu