Re: [Monotone-devel] Re: comments on .sqlite3.binary branch

[Matt Johnston]

On Wed, Jan 18, 2006 at 01:47:41PM +0100, Christof Petig wrote:
> Nathaniel Smith schrieb:
> > Something we should vaguely consider -- switching to 'deflate' instead
> > of 'gzip', since we're rewriting all the packed values anyway.  This
> > saves 18 bytes per cell.  'db info' tells me that in my db, it would
> > save about half a meg (1%).  Not that big a deal, but it might make
> > some code simpler (Matt knows about this).
> 
> Can the inflate code handle both variants or do we need to recode the
> whole database (and introduce yet another binary incompatibility over
> the wire).

New versions of zlib (1.2 and up) can decode gzip data too,
though I don't think we want to force people to get newer
versions.

Recoding the database is relatively straightforward, just
adding an extra helper for the schema migration. It makes
the migration a bit slower, given it has to deflate the
entire db contents. It should be possible to grab the zlib
chunk out of the current gzip encoded data if it's too slow.

The netsync incompatibility is probably more of a hassle in
the short term, though if it were done before 0.26 it
mightn't be too inconvenient.

I've put a diff at
http://matt.ucc.asn.au/monotone/zlib-1.diff 
- it seems to work, probably will require a few changes
before being merged in (testsuite is running now).

The main benefit I see of using deflate/zlib rather than
gzip is that it involves less fiddling around with creating
pipes to calculate yet another checksum (adler32), and also
uses upstream's zlib module. The disadvantage is that you
can't use "gzip -c" to get at the raw data (though 3 lines
of python gives an equivalent tool).

Matt