release process analysis

[Bruno Haible]

Hi Paul,

It's a pity that grep-2.5 was released with such a mistake.

What was the cause, and how could it be avoided?
  * One could say that more testing would have uncovered it. But
    - we have limited testing resources (Jeffrey, Assaf, and I are the only
      people who regularly test on several platforms),
    - we don't have continuous integration on Solaris,
    - the continuous integration on Ubuntu that I am maintaining does not
      include a preinstalled libsigsegv.
  * We send the patches to the list in the hope that they will be reviewed
    even if committed immediately. I usually do review your patches, except
      - when it's code I don't understand (like regex and dfa),
      - when it's too big.
    In this case, I didn't review it because it looked too frightening.
    8ba9126d00bfe1ab77a5c820c58c68933d4df85c contained more than 10 distinct
    changes. It would have been more likely that I review it if it had been
    split across 3 or more patches.

How can we avoid such things?

  (a) By reducing the size / complexity of patches? (Yes, I know, when looking
      at a bunch of old code, it is tempting to make so many improvements at
      once.)

  (b) Introduce a formal checklist of patch reviews? So that at any moment we
      can see which patches have been reviewed and which haven't. And the
      'grep' release manager could use this checklist in order to delay the
      release until the reviews have been completed.

  - or other suggestions?

Bruno