I've figured out why we have bug 1170113, which is that on incremental backups duplicity always reads the latest manifest from the remote even if there is a copy in the local cache. This is usually innocuous, but if the remote is Amazon S3 transitioning to Glacier, the remote can't be read (without a previous arrangements) so it's fatal.
The question is what would be the preferred fix. I'll just copy here my latest entry from the bug report page, and I'd be interested in what you guys think:
I have identified why this bug occurs. For incremental backups duplicity always calls BackupSet.check_manifests(), described as "Make sure remote manifest is equal to local one." It reads the manifest from the remote and checks that it is identical to the local copy. I believe this is the only file that duplicity checks in this way.
Of course, the usual case is that the remote files are encrypted and duplicity is not in possession of the secret key. After reading the file from remote, if duplicity finds this is the case then it just proceeds as if the comparison had succeeded.
As I mentioned above, this behavior is fatal if the remote is Amazon S3 transitioned to Glacier. But there are several ways one could imagine patching it:
1. Get rid of check_manifests() altogether. I don't see why in this one case we should be paranoid about a remote file not matching the one in the local cache.
2. Instead of ignoring only the exception that the secret key is not available, also ignore the exception of not being able to read from the remote.
3. Add a command line flag to disable check_manifests().
- Dave Cottingham