Re: [Duplicity-talk] Support for Amazon Glacier - Amazon Announcement

[edgar . soldin]

On 15.11.2012 23:24, Ed Blackman wrote:
> On Wed, Nov 14, 2012 at 09:44:23PM +0000, Dean Smith wrote:
>> Amazon have announced support for Glacier support via the S3 interface today.
>>
>> I am planning on doing some testing on this but as it can automatically 
>> expire old objects to Glacier I don't think any changes will be necessary 
>> for backup to work.
> 
> My main concern about just letting Amazon expire old objects to Glacier is 
> that naive S3 lifecycle rules ("expire files that are 180 days old") will 
> move *parts* of backup sets to Glacier while keeping other parts in S3.
> 
> My immediate wish for Glacier support in duplicity is for the remove-* verbs 
> to take a --glacier-expire-path option that would take a path under the 
> target URL where the "deleted" files would be moved, rather than being 
> removed.
> 
> I could then just set up a single rule (well, per target URL) that would say 
> ("expire files in <bucket>/older/ after 7 days").  That way duplicity would 
> move an entire backup set there when "removed", and a week later the files 
> would shuffle off to Glacier as a set.
> 
> If I need to restore, I'd just do that manually in AWS Console, and have a 
> week to move the restored files elsewhere.  (I'm presuming that Amazon would 
> restore a file to it's original path.  If not, "7 days" could be changed to 
> "immediately".)
> 
> I could script that myself if there was a way to list all the files 
> associated with a backup set.  I don't see one, but that doesn't mean it 
> doesn't exist.
> 


looks like the S3 to Glacier transition management has a quite elaborate API
http://docs.amazonwebservices.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html
so workarounds like would probably unnecessary, if
- 'remove-' would initiate to move the backup chains directly to glacier via 
e.g. switch '--s3-to-glacier' instead of '--force' for deletion
or
- rules are setup on s3 to transit to glacier e.g. monthly (current-3) with 
corresponding monthly duplicity backup chains

if you do not '--extra-clean' your backups and do not "freeze" your signature 
files you should even be able to list archive contents even though the 
according volumes would still have to be "thawed" if you want to restore.

..ede/duply.net