Re: [Sks-devel] Adding DB_INIT_LOCK to sks-keyserver (revisited)

[Jason Harris]

On Fri, Feb 26, 2010 at 08:20:15PM -0500, Jeff Johnson wrote:
> On Feb 26, 2010, at 8:02 PM, Yaron Minsky wrote:

> > I'm a big fan of sqlite, and Markus Mottl has written an excellent binding 
> > for the library --- far better than the mediocre binding I wrote for bdb.  
> > I think porting sks to use sqlite instead of bdb would be great, although I 
> > would think some thinking about migration would be in order.

Excellent.  Thanks, Yaron.

> I've used both sqlite3 and Berkeley DB, ACID behavior doesn't come for free 
> with
> any implementation, and its ACID which is needed for ~3M keys.
> 
> Here's one datapoint from RPM which has used both BDB and Sqlite3:
>       sqlite3 (after tuning) was 2.4 times slower than BDB (without tuning).
> The straightforward implementation was 14x slower for database intensive 
> operations.

Interesting.  How big were the BlOBs?

> Can you write down the SQL schema currently used in SKS? Every SQL
> implementation starts with the schema ...

DB      DESCRIPTION             SCHEMA
key     raw key blocks          SKS hash -> 1 byte indicator + fastbuild info
                                or       -> 1 byte indicator + actual keyblock

keyid   primary keyids          SKS hash -> pubkey keyids (32 bit keyid)

meta    filters                 literally just the single record:
                                "filters" -> "yminsky.dedup,yminsky.merge"

subkeyid subkey keyids          SKS hash -> subkey keyids (32 bit keyid)

time    shash timestamps        timestamp -> add/delete flag + shash

tqeuue  mail queue              currently empty...

word    userid words            userid word -> SKS hash

-- 
Jason Harris           |  NIC:  JH329, PGP:  This _is_ PGP-signed, isn't it?
address@hidden _|_ web:  http://keyserver.kjsl.com/~jharris/
          Got photons?   (TM), (C) 2004

pgpSZ9BlyO03W.pgp
Description: PGP signature