Re: Please start a release branch (was: CVS 1.11.5 Released

[Larry Jones]

Paul Edwards writes:
> 
> What percentage of bugs, or reported problems, in the last
> year, have been due to bugs that have been undiscovered for
> over 2 years.

>From a quick look at the log, it appears that most of the changes have
been one of two kinds:  fixes of long-standing problems (or fixes of
fixes of long-standing problems that didn't quite work right the first
time), and build system changes.  I would say that most of the recent
instability is not due to bugs in the code per se, but rather to
problems caused by going from a manually generated autoconf script to
one generated by automake and the subsequent restructuring to make that
work in the obvious fashion.  Many of them only affect Windows, which is
not a primary development platform and so the problems don't get noticed
as promptly as problems with the other platforms.  (If anyone would like
to volunteer to run nightly testing on a Windows box, please do!)

> Not sure about the last few years, but cvs 1.10 is a 700k
> executable for me on Solaris.  cvs 1.11.5 is 2.3 meg.  Both
> stripped.

File sizes can be highly misleading; I'd be interested to see what
`size' has to say about the real sizes of those executables.  I'd be
willing to bet that most of that difference is due to different
configuration options and/or different compilers/compile options.  On my
BSD/OS system, CVS 1.11.5 is only about 5% larger than an indentically
configured CVS 1.10 and there's no reason for Solaris to be
significantly different.

> The sort of problems I have seen personally, are:
> 
> 1. Incorrect Makefile linking with some security library which
> wasn't installed.  Switching to --disable-client --disable-server
> got around that.  The security library was a presumed
> enhancement.

That has nothing to do with CVS itself.  Either your system changed and
confused autoconf, or autoconf changed.

> 2. --disable-client --disable-server stopped working.  But by
> this stage the above Makefile had been fixed, so switching
> off these options got around that (for a slightly bigger executable).
> The code causing the problem has since been corrected, but it
> was obviously introduced between 1.11 versions, ie a presumed
> enhancement.  So one problem replaced by another problem, not
> a move to stability, in fact, worse, because it required some
> action to find the workaround to the workaround not working
> around.

--disable-client and --disable-server have not been tested as thoroughly
as they should.  I would say they've been broken more often than they've
worked right.  The fact that they just happened to be working when you
originally used them was a lucky coincidence.

> 3. Procedure for compiling changed.  After having been
> ./configure for x years, I now needed to run noautomake, or else
> I got some error.  Another "enhancement" that could have been
> in 1.12 instead of being introduced between sub-versions of 1.11.

I wasn't real enthusiastic about the change to automake, but I'm
sympathetic to the intent: to make it easier to support new platforms
and to support the existing platforms in a more standard (as opposed to
ad-hoc) manner.  Build problems are usually very obvious and easy to
fix.

> Another problem someone else was having with the Windows
> executable of 1.11.1p1 or somesuch.  Something about userid
> not being known.

That was a side-effect of the move to automake which allowed us to get
rid of a bunch of Windows-specific code.  That's good in the long run,
since the Windows-specific code tends to get ignored and thus doesn't
always get the bug fixes that it should have.  Unfortunately, there have
been a number of problems caused by merging the Windows code into the
base code.  Fortunately, they only affect the Windows code which must
not get used too much since it's taken quite a while for these problems
to be reported.

> Then e.g. there is that bug which I saw you fix of another windows
> user, ndbm or something.  It sounded like it only affected them
> because they updated to the latest greatest version, as opposed to
> the most stable version.  The result of another enhancement?

No, another merge problem.

> So are 90% of the end-user problems attributable to the fact
> that an enhancement being made, or are they due to long
> dormant bugs awakening?  What is the % breakdown?

I don't see *any* end-user problems with existing functionality that's
attributable to what I would consider an enhancement.  I have seen end-
user problems due to missing features, which is what the few small
enhancements have been addressing, and I've seen end-user problems with
an enhancement itself not working right, but essentially all of the
other end-user problems have been ultimately attributable to long-
standing bugs, in my opinion.

> Because at the moment, whenever I get a new drop of CVS,
> my heart stops beating as I wonder whether it will even
> compile on my system.  I basically expect to be hit with
> some enhancement that is so fundamental it doesn't even
> allow me to compile.  I currently have no expectation that
> when I go from 1.11.3 to 1.11.4 I will go from strength to
> strength, I instead have an expectation that I will be
> replacing one set of bugs with an equal number of bugs.

Then your expectations are completely wrong.  Whether it compiles or not
has almost nothing to do with the quality of the code.  As I explained
above, compile failures are almost always due to changes in the
configuration and build process rather than any enhancements in the CVS
code itself.  They are usually quite obvious (e.g., compilation fails)
and easy to fix.  The overall code quality has, in my opinion, been
monotonically increasing for a long time.  With few exceptions, I always
recommend newer releases over older releases, and I usually even
recommend the current development version over the latest release.  The
few exceptions there are tend to be embarassing show-stopper type
problems that only affect certain environments.  Note that we run the
entire test suite, in both local and client/server modes, on the current
development version of the code every night on about a half-dozen
different platforms, so it's quite difficult for anything to get broken
and not noticed very quickly.

> If the small features aren't introducing a plethora of bugs,
> all is fine.  So long as 10 bugs get swatted for every 1
> bug introduced, that would be reasonable, as it means
> we're on the track to stability.  At the moment I can't even
> see the track.  This is just my personal experience.  Maybe
> I'm the odd one out.  Maybe I'm the one who experiences
> the 1 bug introduced for 10 swatted.

I think you've just noticed a lot of teething pains in a new build
system and incorrectly generalized that into a belief that the mainline
code is unstable.  I'd say we're doing a lot better than 10 fixes for
each new bug in the mainline code.

-Larry Jones

Physical education is what you learn from having your face in
someone's armpit right before lunch. -- Calvin