Re: [Monotone-devel] Hash collisions resiliency

[J C Lawrence]

On Wed, 13 Apr 2005 21:20:57 +0200
Jon Bright <address@hidden> wrote:
> J C Lawrence wrote:

> But if something's so incredibly improbable that a hardware failure
> whilst also winning the lottery is much, much more likely, it's not
> worth thinking about.

I disagree, mostly as it fails to distinguish between "the probability
of it happening to me" versus "the probability of it happening at all".
In facetious contrast I would offer:

  Consider a useful and popular household device, say a dish washer.
  This particular version washes dishes very efficiently and well, but
  has a 2^50 chance on any given dish in a wash cycle of going nuclear
  and taking out your city block.

While a rather laughable argument, some source trees have estimated
values in that range and thus encourage a similar sort of risk
management.

> Or are you proposing that Monotone also try to detect hardware
> failures in a way other than throwing invariant failures showing their
> effects?

Nope, what I'm hoping for is that failures, like improbable hash
collisions, are detected, absolute and very very clear.

> You could have monotone try to check for SHA collisions when adding
> files or revisions.  This would slow things down by quite a bit since
> it'd be necessary to check file contents against one another rather
> just checking if the SHA hashes match.  You could check file lengths
> against one another - this wouldn't slow things down that much.

That latter form seems like a reasonable first-order optimisation.  If
the hashes match and the lengths are the same, then go into full
check-for-collision mode.

>> To be specific: I or someone on the team would have to notice these
>> facts through simple manual observation of some sort of unexpected
>> behaviour, or would there be the equivalent of Bubba the Neanderthal
>> whacking me upside the head with a clue-by-four and yelling, "HEY
>> BOZO, YOU HAVE A HASH COLLISION!"?

> I can't say which, but you'd probably hit some invariant failures of
> some sort at some point pretty soon afterward.  These wouldn't say
> "hash collision", but they would say you have something pretty broken
> in your DB.

<nod>  At the end of the day there are two concerns:

  1) Will I know that something broke?

  2) Can I recover and at what expense?

As long as the answer to the first one is "Yes" and is immediately after
the actual first error, then the second one has a chance of being
managed, if only by restore-a-backup.

--
J C Lawrence
---------(*)                Satan, oscillate my metallic sonatas.
address@hidden               He lived as a devil, eh?
http://www.kanga.nu/~claw/  Evil is a name of a foeman, as I live.