|
| From: | Russell Clemings |
| Subject: | Re: [Duplicity-talk] Create full backup from incremental |
| Date: | Fri, 17 Apr 2015 07:04:21 -0700 |
+1 for the incremental against an older full, we could call it synthetic chain or rebased or?
of course the purge routines would have to be touched to respect such dependencies as well.
..ede/duply.net
On 17.04.2015 14:35, Kenneth Loafman wrote:
> It's way off the roadmap for now, but a separate utility to accomplish this might be workable. I can't help but think that the space and network requirements are going to be the same as that of a full backup or restore.
>
> Rather than doing that, why not implement a backup where instead of doing an incremental based on the current chain, you reset the incremental process and start over with a second chain off the same full. I have no name for that, and actually just thought of it, but it would solve some of the problems that people seem to have with backups. That way you reuse your base full backup and still have incrementals.
>
> ...Ken
>
>
> On Fri, Apr 17, 2015 at 4:07 AM, Edgar Soldin <address@hidden <mailto:address@hidden>> wrote:
>
> Ken? s.b. ..ede
>
>
> -------- Forwarded Message --------
> Subject: Re: [Duplicity-talk] Create full backup from incremental
> Date: Thu, 16 Apr 2015 18:16:18 -0600
> From: Eric O'Connor <address@hidden <mailto:address@hidden>>
> Reply-To: Discussion of the backup program duplicity <address@hidden <mailto:address@hidden>>
> To: address@hidden <mailto:address@hidden>
>
> Ya, I think it would take some work to make this happen, but I don't
> think duplicity's approach is incompatible. Most tricky would be
> to allow incremental backups based on a syn-full, as you mention in your
> second bullet point.
>
> I wouldn't consider the current backup chain (full + incrementals) to
> have similar properties to a "full" backup, synthetic or otherwise.
> Recovering the most recent state takes a bunch of processing time and
> extra storage/bandwidth, both of which grow with the length of the chain.
>
> Would you be interested in patches to implement this, or is it too far
> off the roadmap?
>
> Eric
>
> On 04/16/2015 02:59 AM, address@hidden <mailto:address@hidden> wrote:
> > thx Eric, unfortunately the current duplicity design is such as that
> > - it bundles changes to different files in a volume until max size
> > and than continues in a next volume - the changes are rsync diffs
> > that have to be applied in a row eg. first state will be restored,
> > first rsync diff will be applied, second rsync.. etc. until the
> > latest state is restored
> >
> > if i understood your explanation correctly than this would mean that
> > currently our "synthetic full" is essentially our complete backup
> > chain.
> >
> > ..ede/duply.net <http://duply.net>
> >> On 2015-04-15 13:48, address@hidden <mailto:address@hidden> wrote:> >
> > On 15.04.2015 23:24, Eric O'Connor wrote:
> >> For this feature, the remote doesn't really need to have access to
> >> the data, or be very smart at all (dumb file servers work just be
> >> fine). It is true that Duplicity does not support it yet.
> >>
> >> Doing a synthetic full backup requires only that you be able to
> >> (locally) keep track of where on the dumb server the latest
> >> version of each file is stored, and which files are recently
> >> modified. Then a full backup is the set of archive files containing
> >> unmodified files (likely a large percentage) + new archives
> >> containing files modified since the last syn-full. So you upload
> >> the new archives, and an index pointing to all the relevant data
> >> chunks.
> >>
> >> It's a true "full backup" because it directly contains every file
> >> needed to do a restore.
> >>
> >> When an archive file contains 1 file, there is no additional data
> >> storage overhead to this -- you just upload a new index and all the
> >> modified files. If archive files contain more than 1 file, a full
> >> backup will have some storage overhead -- some files in the
> >> archives will be irrelevant older copies. The backup program can
> >> pick some overhead maximum and upload enough new data to reduce
> >> the overhead to acceptable levels.
> >>
> >> This can also be spread out over the course of the full backup
> >> period -- i.e. every day upload an incremental backup along with
> >> 5% of the modified files. You could also occasionally re-upload
> >> unmodified files such that it's more likely a single archive
> >> corruption is recoverable. It may even be possible to ditch the
> >> full/incr schedule entirely if the length of an incremental chain
> >> for a file has an upper bound.
> >>
> >> Anyway, sorry for being pedantic, unhelpful? I've been thinking
> >> about building something like this for a while but haven't gotten
> >> around to it yet. Also, duplicity works well enough -- so thanks
> >> for that :)
> >>
> >> Eric
> >>
> >>> good point.. why would you need encrypted backup if you trust the
> >>> backend?
> >>>
> >>> thx Scott.. ede/duply.net <http://duply.net>
> >>>
> >>> On 15.04.2015 19:30, Scott Hannahs wrote:
> >>>> Note, that to do this, you need to have unencryption locally on
> >>>> the server. Duplicity assumes an insecure server model. To
> >>>> collapse incremental backups onto a full backup means that all
> >>>> your data is compromised to the level of security of the remote
> >>>> server.
> >>>>
> >>>> The duplicity model assumes that once the data goes out over
> >>>> the wire it is subject to unknown security.
> >>>>
> >>>> For any commercial remote storage, you might as well just use a
> >>>> commercial backup system without encryption.
> >>>>
> >>>> -Scott
> >>>>
> >>>> On Apr 15, 2015, at 07:21, address@hidden <mailto:address@hidden> wrote:
> >>>>
> >>>>> On 15.04.2015 12:56, Ulrik Rasmussen wrote:
> >>>>>> On Wed, 15 Apr 2015 12:00:00 +0200 address@hidden <mailto:address@hidden>
> >>>>>>> ..ede/duply.net <http://duply.net>> >>>>>> wrote:
> >>>>>>
> >>>>>>> On 15.04.2015 09:54, Ulrik Rasmussen wrote:
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I just started using duplicity for backing up my work
> >>>>>>>> to a VPS. It is my understanding that it is wise to do
> >>>>>>>> a full backup about once a month, to enable deletion of
> >>>>>>>> old backups and faster restoration. However, when doing
> >>>>>>>> a full backup, duplicity seems to transfer everything
> >>>>>>>> over the wire again, which takes a long time if I'm on
> >>>>>>>> a slow connection and also costs me bandwidth. Since
> >>>>>>>> the server already has all my data, this really
> >>>>>>>> shouldn't be necessary.
> >>>>>>>>
> >>>>>>>> Is there a way to do a full backup on the server side?
> >>>>>>>> More precisely, can I tell duplicity to create a new
> >>>>>>>> backup chain based on the contents of the current
> >>>>>>>> chain?
> >>>>>>>>
> >>>>>>>
> >>>>>>> no.
> >>>>>>>
> >>>>>>> duplicity deals with "dumb" backends and solely uses them
> >>>>>>> for file storage. for this design to create a synthetic
> >>>>>>> full you would have to transfer the whole data over the
> >>>>>>> line again anyway completely.
> >>>>>>>
> >>>>>>> however, it'd be possible to implement that for the rare
> >>>>>>> cases that users have shell access to their backends
> >>>>>>> and can have a duplicity instance running locally there.
> >>>>>>>
> >>>>>>> see also
> >>>>>>> https://answers.launchpad.net/duplicity/+question/257348
> >>>>>>>
> >>>>>>
> >>>>>> I see, thanks for clarifying. That makes sense, considering
> >>>>>> most backends don't imply shell access. Since I _do_ have
> >>>>>> shell access to the server and plenty of disk storage, I
> >>>>>> guess I can accomplish the task by just restoring the
> >>>>>> incremental backup on the server and doing a full backup
> >>>>>> from that using the file system backend.
> >>>>>>
> >>>>>
> >>>>> right you are.. make sure to have identical users/numeric ids
> >>>>> and restore as root, if you want to keep those.
> >>>>>
> >>>>> alternatively you can hackishly "reuse" the old full by
> >>>>> copying it and updating the filenames with a proper newer
> >>>>> timestamp. depending on your data's turnover you might be
> >>>>> doing that for a while until your first incremental grows
> >>>>> too big.
> >>>>>
> >>>>> ..ede/duply.net <http://duply.net>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Duplicity-talk mailing list address@hidden <mailto:address@hidden>
> >>>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >>>>
> >>>>
> >>>> _______________________________________________ Duplicity-talk
> >>>> mailing list address@hidden <mailto:address@hidden>
> >>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >>>>
> >>>
> >>> _______________________________________________ Duplicity-talk
> >>> mailing list address@hidden <mailto:address@hidden>
> >>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >>>
> >>
> >>
> >> _______________________________________________ Duplicity-talk
> >> mailing list address@hidden <mailto:address@hidden>
> >> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
> > _______________________________________________ Duplicity-talk
> > mailing list address@hidden <mailto:address@hidden>
> > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
> s
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden <mailto:address@hidden>
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk
| [Prev in Thread] | Current Thread | [Next in Thread] |