[Duplicity-talk] (Option to) Cache retrieved volumes between (fetching/restoring) runs locally?!

[Daniel Hahler]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello,

I would like to be able to cache retrieved files from the backend
locally between multiple runs of duplicity, e.g. via some config or
command line option.

Use case: having accidentally overwritten a lot of my (virtual)
containers files, I've used the following to restore the previous state:
  for i in $=VCIDS; do
    b=path/to/files
    /bin/rm -rf /$b*
    duply profile fetch $b /$b 1D
    duply profile fetch ${b}.d /${b}.d 1D
  done

This makes up 60+ runs of duplicity (2 runs of duplicity per 30+
containers, one for a single file, the other for a directory), and when
looking at it with "--verbosity 9" it looks like a lot of the same
volumes (with a size of 50M in my case) are downloaded every time.

I think it would speed this (particular) use case dramatically up, if
these files would get cached locally.

I could imagine to configure something like "keep files for X hours",
and when duplicity gets run and files are older than this, they get
cleaned on shutdown.
When being accessed, they would get touched to reset the timer.

However, there should also be a maximum number of files to cache, since
this might easily fill your local volume otherwise.

I am thinking about caching the files encrypted (just as on the remote
site), but maybe caching decrypted files would make sense, too?

Obviously, this should take into account if this is a remote backup
(maybe by looking at the transfer rate of the files?!), and not pollute
the cache, if the backend is as fast as local transfers might be.

What do you think?


Cheers,
Daniel
- -- 
http://daniel.hahler.de/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFM2bSrfAK/hT/mPgARAmi8AKC8BIagtiG5iMEoPVLy5spcDoUXLwCfRS9+
fb74hACjTAaNMItAZSJnMd8=
=Oe6l
-----END PGP SIGNATURE-----