[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] Deleting large files from archive?
|
From: |
David Schneider-Joseph |
|
Subject: |
Re: [Duplicity-talk] Deleting large files from archive? |
|
Date: |
Tue, 29 May 2007 11:55:34 -0400 |
Kenneth and Charles, thanks for your thoughts. What do you think of
Gabriel's idea? For a 1 GB file, would you be able to do a blind
delete of 203 5 MB chunks, and possibly only have to edit 2 of the
remaining 5 MB chunks?
On May 28, 2007, at 2:52 PM, Gabriel Ambuehl wrote:
On Monday 28 May 2007 20:26:55 David Schneider-Joseph wrote:
Presumably this is technically feasible, as the --remove-older-than
option manages to selectively delete only those files that are no
longer referenced, but it seems that duplicity does not currently
support this. How would I go about doing this with the underlying
rdiff tools?
Technically it seems feasible to delete all tar chunks that contain
ONLY THAT
ONE file. So for a big file, this might work. However, you would
have to
somehow make clear to the restore process that the left over chunks
in the
first and last tar chunks (which almost always will be shared with
other
files and thus cant be removed) are to be discarded for it not to get
confused.
As for implementation, I don't know enough about rdiff to comment
on that.
On May 29, 2007, at 10:44 AM, Kenneth Loafman wrote:
David Schneider-Joseph wrote:
To follow up: I consider this to be a major missing feature.
There's absolutely nothing to be done, and no future course for
development? If you could point me in the right direction I'd be
willing to work on a patch myself.
The problem is that to modify a file on the remote system requires
that you download that file locally, modify it, then send it back.
Since this is a multi-file tar archive, you would need to download
the entire archive, decrypt, uncompress, delete the file, compress,
encrypt, then upload the modified archive, making sure to delete
the unused portions.
The remote system is *just* storage. It has no processing
capabilities w.r.t. the archive itself.
...Ken
On May 29, 2007, at 10:54 AM, Charles Duffy wrote:
David Schneider-Joseph wrote:
To follow up: I consider this to be a major missing feature.
There's absolutely nothing to be done, and no future course for
development? If you could point me in the right direction I'd be
willing to work on a patch myself.
Duplicity's design pretty much precludes it; consequently, a
storage format change would be necessary to support this feature,
and storage format changes are... a significant undertaking.
rdiff-backup also doesn't have this feature, though its design is
much more amenable to the extension (and history editing is a much-
requested thing there as well) -- but if you need duplicity's
security guarantees, the lack of a straightforward mechanism for
implementing reasonable/efficient history editing (without a
complete disk format rework -- which would need to be done /very/
carefully to maintain duplicity's present security guarantees) is
the consequence.
I can think of a few ways to modify the disk format to maintain
reasonable security guarantees and place an upper bound on the
amount of information that needs to be transmitted back and forth
during the history-editing process -- but anything that comes to
mind would require larger code changes than are presently feasible,
never mind the complete lack of backwards compatibility.