[rdiff-backup-users] Rewriting metadata to use arbitrary storage system (SQLite)?

[rdiff]

I have been giving thoughts to breaking rdiff-backup's meta-data code into 
a normalized abstraction to unlink rdiff-backup from its meta-data 
storage.  This would allow meta-data to be stored in an SQL server, DBM, 
flat file (current) or quantum goo (future?).

Creating meta-data interfaces for different SQL implementations such as 
SQLite or postgresql/mysql for bigger centralized systems could offer the 
flexibility which rdiff-backup really needs.  The ability to run queries 
on the meta-data might also give us the flexibility to detect directory 
renames.  Also think here about commit/rollback for real ACID transaction 
compliance (SQL junkies can druel here ;).  This would truly increase the 
solidity of rdiff-backup.

******** With this in mind, I propose that we release a "stable" release 
and apply bugfixes to it as appropriate.  The latest release seems fairly 
stable according to the list.  Is it time to move 1.1.x to 1.2.x and 
create a new 1.3.x development fork for the more drastic changes which 
people are beginning to need. ********

Directory rename detection would be an AWESOME feature.  We have one 3TB 
repository and several >100GB repositories backing up over the Internet 
with anywhere from 1GB to 100GB of changes per night. When someone renames 
a 500GB folder it (1) takes time to figure out that is what happened and 
(2) more time to fix the problem by either (a) blowing away the 
increments, renaming the folder by hand and doing an initial sync or (b) 
driving the backup server over to the system-to-be-backed up and re-sync 
everything.

FYI, at 60mbit, the 3TB backup took 6 days to do an initial sync!  It was 
a gigabit link, but the system we were pulling from wasn't fast enough to 
utilize the whole pipe, and we sat at ~60mbit.  I am pleased to announce 
that it is quite stable though, now that the initial sync has completed 
which does say something for rdiff-backup's scalability!

Thoughts?

-Eric