On Sun, 2009-09-13 at 19:19 +0300, Avi Kivity wrote:
On 09/10/2009 11:36 PM, Mark McLoughlin wrote:
On Thu, 2009-09-10 at 21:40 +0300, Avi Kivity wrote:
(and we're quite far from catching every regression btw).
That's kind of my point. A lot of regressions are only found after
they've been pushed. Delaying a push delays finding those regressions.
That makes sense if the regression tests don't find any regressions.
Or if the regression tests take too long - rather than regression tests
that take 5 hours to run, I'd prefer to see a subset of those which
takes e.g. 30 minutes be used to sanity check a tree before pushing it.
If
they do, then pushing despite those regressions means everyone is now
hitting the regressions, swearing, and trying to work around them. We
get massive parallelism but little progress.
Not all regressions are equal. I wouldn't mind some regressions in the
tree so long as they are being tracked as "must be fixed" before the
release.
Clearly anything causing people to swear should be reverted. Assuming
you can tell which patch caused the problem.
I think reducing the batch size will also help. If the batch contains
100 patches there's a high likelihood of a regression in there. If
you're testing 10-15 patches at a time it may actually work, and if not,
you can easily guess who the offender is.
The batch size is determined by the length of time the testing takes,
right?
But agree, with a large batch, you're more likely to find at least one
regression, to struggle to figure out which patch caused the regression
and to go through several iterations before you have something to push.