Re: [Monotone-devel] Bugs, features and history (was Re: pretty pretty pictures (for some value of pretty))

[Rob Schoening]

I think the key point that I was trying to make and I seems that Koen was saying as well, was that the program doesn't suck and we're not really grumpy at all! 

 

The roadmap that Nathaniel and others have presented in this thread seems entirely rational. 

 

To my mind this is one of the better-run open source projects I've seen.  I'm just trying to set my own expectations of where it might be a year or two out.

 

RS
 

On 9/15/06, Graydon Hoare <address@hidden> wrote:

Koen Kooi wrote:

> I still think monotone is the best open-source VCS/SCM around, but I am a bit dissapointed
> at 'history digging' tools and their performance. And more recently in 'mtn add' and 'mtn
> diff' being broken by the IMO bogus restriction code. So I'm wondering, is there a plan to
> fix bugs like this, or is the plan to keep adding more features like policy branches and
> do a bug-sprint after that?

I think it's a mistake to view policy branches as "new feature" work.
It's new vocabulary, but it's related to a concrete cluster of bugs. I
want to draw your attention to history for a moment.

Three and a half years ago the biggest "foreground" cluster of bugs
looked like this:

  - The NNTP protocol can get wedged on some news servers.
  - The HTTP protocol can get wedged on some web servers.
  - Depots can get desynchronized from clients such that receive order
    is wrong.
  - Queueing packets for transmission can go wrong and send stuff that
    gets silently dropped.
  - It's not clear how to handle new depots that share a bunch of
    history. Partial re-send?

And we were miserable and grumpy and convinced the program sucked, but
then sobered up and looked and saw the problems all centered around a
bunch of code that's completely forgotten now. So we sat around
discussing set-theoretic problems for a while, and bloom filters and
merkle codes and stuff. We finally decided that on-the-fly
synchronization was a better deal, cut out the networking and
packet-queueing code, and replaced it with netsync.

Two and a half years ago the biggest "foreground" cluster of bugs looked
like this:

  - The merger and other history-tracing code can get lost in history
    cycles.
  - Two projects can have their histories conjoined by accident.
  - Malicious users can time-travel.
  - File identity is being inferred in an unreliable way.
  - Renaming things produces certs, so file identity depends on
    trust hooks?

And we were miserable and grumpy and convinced the program sucked, but
then sobered up and looked and saw the problems all centered on
patch_set.cc (also now long dead). So we sat around discussing
composable change algebras and immutable histories and iterated hashing
and stuff. We finally decided that explicit, linked history logs were a
better deal, cut out the implicit patch_set code and replaced it with
change_sets and revisions.

A year and a half ago the biggest "foreground" cluster of bugs we had
looked like this:

  - Merge can go crazy and produce illegal history.
  - Moving directories can cause crashes.
  - Restrictions get confused and include or exclude the wrong files.
  - Independent adds can cause irreconcilable histories that just crash
    the merger.
  - Sometimes the merger silently succeeds and trashes your work when
    it should not.

And we were miserable and grumpy and convinced the program sucked (even
Linus said so!), but then we sobered up and looked and saw the problems
all centered on change_set.cc and the godawful merge "algorithm" in
there (I use this term loosely), and to a lesser extent on the continued
use of manifest.cc. So we sat around discussing graph theory and user
models and file identity stuff. We finally decided that a
non-transmitted cache of file identities and cached
quasi-graph-dominance information was a better deal, cut out the
change_set and manifest code and replaced it with cset, roster and
multi-mark-merge.

Now the biggest "foreground" cluster of bugs looks like this:

  - Need to rename branches
  - Need to mark branches as "done" so they stop polluting UI
  - Need to rename keys
  - Need to kick users out of projects occasionally
  - Need a way to denote "trusted core developers" and similar
    subsets of devs (translators, etc.) in a simple way
  - Need a way to migrate to ever-stronger crypto as algorithms break
  - Need a way to automatically set up new users with the trust rules
    of a project, without having them edit lua files.

A few things stand out here: first that these bugs are actually all
*very old* bugs that have been deferred for years, second that they're
not so much breakage as "incorrect initial implementation" so they smell
a bit more like features, and third that they're all relatively shallow
and shouldn't even require a history-graph rebuild once we figure out
the right structure. That's *fantastic* news to me; it suggests we're
actually making the set of bugs smaller over time. It almost restored my
faith in progress!

Following the pattern, I think we've passed the misery and drinking
stage, so I expect a flurry of discussion on theory for a while, then an
experiment that cuts out the rotten old parts (hint: cert.cc, keys.cc,
update.cc) and replaces them with something conservatively reflecting
current thought and experience, followed by a release that solves a
whole whack of problems at once. And as the ROADMAP file has indicated
for some time, I think this might actually be one of the last big
cut-and-replace jobs (merge-into-dir being the other, but it seems to be
happening concurrently).

Does this mean that there's less interest in more isolated, one-off
bugs? Possibly a bit, but no more than the historical average.
Everyone's attracted to big-bang fixes. They're dramatic. But I don't
think that fact represents a strategic shift away from "addressing
bugs". That's still the primary motivation. We'll have another sprint to
gather up the forgotten, less glamorous one-off bugs someday too.

-graydon



_______________________________________________
Monotone-devel mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/monotone-devel