Re: [Gnu-arch-users] a brief note on "delta compression"

[Martin Pool]

On  3 May 2004, Tom Lord <address@hidden> wrote:

>     > So we have to pick an X that is an estimate of how many revisions people
>     > are missing at any given time. Too high, and these extra stored
>     > revisions go unused, because nobody is that far away from the most
>     > current revision to use the CD. Too low, and we're not gaining the
>     > full benefit of storing extra CDs. 
> 
>     > My personal hunch is that the proper X is right around 5. I've got no
>     > actual proof for that, mind you. 

Another approach is to have a separate branch which merely bundles up
changes into larger/more efficient packets.  I could create a robot
which every night just merges Tom's changes onto a branch, commits,
and cacherevs.  Then anyone will be able to get either a snapshot of
Tom's latest work, or to update from yesterday, with only a single
request.

I could also stagger them with weekly and monthly snapshots.  So
updating from, in turn, tla--monthly-bundle--3, tla--weekly-bundle--3,
tla--daily-bundle--3, tla--devo--3 ought to bring you up to date in ~4
changesets.

This is kind of similar to how with the linux kernel you might need to
get all the 2.6.1 patches, then some pre patches, then some patches
from a port maintainer to get to the tree tip that you want.

Possibly this makes visible something which ought to be an
implementation detail.  On the other hand I think it's kind of cool
that it can be done with no new mechanism.

One drawback is that you need to trust me not to introduce changes
that weren't in Tom's source.  But this seems to be a general problem
with smart servers or anything else that lets people download anything
but the exact bits that were uploaded.  They won't get any signature
to indicate that it's identical.  Possibly Arch could borrow some
ideas from Monotone to do that kind of transitive authentication.

> Yet another idea, for smart servers, is to cache custom-made summary
> deltas.   The protocol design discussion we had a while back touched
> on this.   The idea is:  suppose that 80% of the readers of the tla
> archive last updated at patch-K.   Recently I announced a new release
> at patch-N.    Many people need summary patch-K...patch-N.   If a
> smart-server computes that and caches it on demand.....

I'd just like to point out that at least at this level, the smart
server can be done without needing a new network protocol.  It'd be
pretty easy for an http cgi program to make new cacherevs magically
appear the first time they're requested.  I'd like to write such a
thing eventually.

-- 
Martin

signature.asc
Description: Digital signature