Re: [Duplicity-talk] why restart takes so long

[Kenneth Loafman]

Do you have a lot of very small files?  This should not take too long.

On Tue, May 10, 2016 at 12:02 PM, <address@hidden> wrote:

Well, what happens is that it says A and all the filenames, but no
reference to a tar file, so its not really backing up again, this is why
I was wondering what it is doing.  The part of the program which seems
to be involved is this:
def restart_position_iterator(tarblock_iter):
    """
        Fake writing to backend, but do go through all the source paths.
               Stop when we have processed the last file and block from the
                      last backup.  Normal backup will proceed at the start of
                         the
                             next volume in the set.

    @type tarblock_iter: tarblock_iter
        @param tarblock_iter: iterator for current tar block

    @rtype: int
        @return: constant 0 (zero)
               """
                      last_index = globals.restart.last_index
                             last_block = globals.restart.last_block
                                    try:
                                            # Just spin our wheels
                                                          iter_result =
        tarblock_iter.next()
                   while iter_result:
                                     if (tarblock_iter.previous_index ==
        last_index):
                           # If both the previous index and this index are
        done, exit now
                           # before we hit the next index, to prevent
        skipping its first
                           # block.
                                                    if not last_block and not
        tarblock_iter.previous_block:
                               break
                                                           # Only check block number if
        last_block is also a number
                           if last_block and tarblock_iter.previous_block >
        last_block:
                               break
                                                       if tarblock_iter.previous_index
        > last_index:
                           log.Warn(_("File %s complete in backup set.\n"
                                                               "Continuing restart
        on file %s.") %

Kenneth Loafman <address@hidden> wrote:

> It sounds like the backup restarted completely, rather than at the last
> known spot.  What do the first 50 or so lines of the log contain?
>
> ...Ken
>
>
> On Tue, May 10, 2016 at 10:48 AM, <address@hidden> wrote:
>
> > But the  question is why does duplicity need to do that at all?
> > Couldn't it go through the last hundred or so or some reasonable number?
> > Also, what about the original error which caused the restart in the
> > first place?  What is up with that that?
> >
> >
> > Kenneth Loafman <address@hidden> wrote:
> >
> > > No, there is no way to get rid of the loop, whichever one you're talking
> > > about, there are a lot of them.
> > >
> > > The main suggestion I have for anyone with multi-hour backups is to:
> > >
> > >    1. Use a local filesystem or USB drive for backup,
> > >    2. Transfer the backup using rsync or similar tool.
> > >
> > > The second suggestion I have is to chunk the backup into reasonably sized
> > > portions, 200-500GB each.
> > >
> > > ...Ken
> > >
> > >
> > > On Tue, May 10, 2016 at 7:02 AM, <address@hidden> wrote:
> > >
> > > > Hi.  I had to restart my full backup and its going through every file
> > it
> > > > had and it is taking 18 hours and its not done yet.  I see a loop in
> > > > there and I wonder if there is any way to shorten the time or get rid
> > of
> > > > the loop altoggether?
> > > >
> > > > Thanks in advance for any suggestions.
> > > >
> > > > --
> > > > Your life is like a penny.  You're going to lose it.  The question is:
> > > > How do
> > > > you spend it?
> > > >
> > > >          John Covici
> > > >          address@hidden
> > > >
> > > > _______________________________________________
> > > > Duplicity-talk mailing list
> > > > address@hidden
> > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > >
> > >
> > > ----------------------------------------------------
> > > Alternatives:
> > >
> > > ----------------------------------------------------
> > > _______________________________________________
> > > Duplicity-talk mailing list
> > > address@hidden
> > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
> > --
> > Your life is like a penny.  You're going to lose it.  The question is:
> > How do
> > you spend it?
> >
> >          John Covici
> >          address@hidden
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
>
> ----------------------------------------------------
> Alternatives:
>
> ----------------------------------------------------
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk

--
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         address@hidden

_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk