How do I get this branch, I have installed this on gentoo using a tar
archiv in the download directory. Can you give me the patch, so I can
fix by hand?
Also, I wonder why an upload should fail at all since there is no space
limit?
Thanks much.
Scott McKenzie <address@hidden> wrote:
> Hi covici
>
> I am not sure why the restart is taking so long, but I found a bug in the
> azurebackend that was causing it to tail occasionally. Instead of retrying
> a failed upload it would cause duplicity to fail the backup. You can find
> the fix here:
> https://code.launchpad.net/~noizyland/duplicity/fix_azurebackend_typo
>
> Please test it and let me know if it helps.
>
>
> -Scott
>
>
> On Thu, 12 May 2016 at 02:25 <address@hidden> wrote:
>
> > Well, here are the first few lines from the log when I did the restart.
> > Since I can't see my own posts, I can't answer the question about the
> > traceback.
> >
> > Using archive dir: /root/.cache/duplicity/backup_linux
> > Using backup name: backup_linux
> > Import of duplicity.backends.acdclibackend Succeeded
> > Import of duplicity.backends.azurebackend Succeeded
> > Import of duplicity.backends.b2backend Succeeded
> > Import of duplicity.backends.botobackend Succeeded
> > Import of duplicity.backends.cfbackend Succeeded
> > Import of duplicity.backends.copycombackend Succeeded
> > Import of duplicity.backends.dpbxbackend Failed: No module named dropbox
> > Import of duplicity.backends.gdocsbackend Succeeded
> > Import of duplicity.backends.giobackend Succeeded
> > Import of duplicity.backends.hsibackend Succeeded
> > Import of duplicity.backends.hubicbackend Succeeded
> > Import of duplicity.backends.imapbackend Succeeded
> > Import of duplicity.backends.lftpbackend Succeeded
> > Import of duplicity.backends.localbackend Succeeded
> > Import of duplicity.backends.mediafirebackend Succeeded
> > Import of duplicity.backends.megabackend Succeeded
> > Import of duplicity.backends.multibackend Succeeded
> > Import of duplicity.backends.ncftpbackend Succeeded
> > Import of duplicity.backends.onedrivebackend Succeeded
> > Import of duplicity.backends.par2backend Succeeded
> > Import of duplicity.backends.pydrivebackend Succeeded
> > Import of duplicity.backends.rsyncbackend Succeeded
> > Import of duplicity.backends.ssh_paramiko_backend Succeeded
> > Import of duplicity.backends.ssh_pexpect_backend Succeeded
> > Import of duplicity.backends.swiftbackend Succeeded
> > Import of duplicity.backends.sxbackend Succeeded
> > Import of duplicity.backends.tahoebackend Succeeded
> > Import of duplicity.backends.webdavbackend Succeeded
> > Reading globbing filelist /etc/azure_excludes.txt
> > Main action: inc
> >
> > ================================================================================
> > duplicity 0.7.07 (April 10, 2016)
> > Args: /usr/bin/duplicity -v info --volsize 300 --gpg-options
> > --pinentry-mode loopback --exclude-filelist /etc/azure_excludes.txt
> > --name backup_linux --encrypt-key address@hidden / azure://linux
> > Linux ccs.covici.com 4.1.17-gentoo #1 SMP PREEMPT Mon Feb 15 15:05:32
> > EST 2016 x86_64 Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz
> > /usr/lib/python-exec/python2.7/python2 2.7.11 (default, Jan 29 2016,
> > 22:32:51)
> > [GCC 4.9.3]
> >
> > ================================================================================
> > Using temporary directory /tmp/duplicity-K67URk-tempdir
> > Temp has 37554356224 available, backup will use approx 408944640.
> > Local and Remote metadata are synchronized, no sync needed.
> > Last full backup left a partial set, restarting.
> > Last full backup date: Fri May 6 16:24:16 2016
> > RESTART: Volumes 45955 to 45955 failed to upload before termination.
> > Restarting backup at volume 45955.
> > Deleting /tmp/duplicity-K67URk-tempdir/mktemp-gLMnDl-2
> > Restarting after volume 45954, file
> > var/www/
> > covici.com/htdocs-secure/owncloud/data/mattguice/files/20151114Town
> > Hall Meeting (Choral Prelude).mp3, block 977
> > A .
> > A audio
> > A audio/.mp3crc
> > A audio/0015.ps
> >
> > And so on and so forth.
> >
> > Kenneth Loafman <address@hidden> wrote:
> >
> > > One quick question about the format of the traceback you show... is that
> > > because of the email client you use, or does it show up that way on the
> > > console?
> > >
> > > I'd like to see the block of data at the first of the log, say the first
> > > 200 lines. Please dump in raw text format.
> > >
> > >
> > > On Tue, May 10, 2016 at 2:19 PM, <address@hidden> wrote:
> > >
> > > > I have thousands of files, some large and some small, its about a
> > > > terabytes worth. It stopped pretty near the end and it just stopped
> > > > again, so my emphasis wuld be to fix the error and maybe do something
> > > > about optimizing the program.
> > > >
> > > >
> > > > Kenneth Loafman <address@hidden> wrote:
> > > >
> > > > > Do you have a lot of very small files? This should not take too
> > long.
> > > > >
> > > > >
> > > > > On Tue, May 10, 2016 at 12:02 PM, <address@hidden> wrote:
> > > > >
> > > > > > Well, what happens is that it says A and all the filenames, but no
> > > > > > reference to a tar file, so its not really backing up again, this
> > is
> > > > why
> > > > > > I was wondering what it is doing. The part of the program which
> > seems
> > > > > > to be involved is this:
> > > > > > def restart_position_iterator(tarblock_iter):
> > > > > > """
> > > > > > Fake writing to backend, but do go through all the source
> > > > paths.
> > > > > > Stop when we have processed the last file and block
> > > > from the
> > > > > > last backup. Normal backup will proceed at
> > the
> > > > > > start of
> > > > > > the
> > > > > > next volume in the set.
> > > > > >
> > > > > > address@hidden tarblock_iter: tarblock_iter
> > > > > > address@hidden tarblock_iter: iterator for current tar block
> > > > > >
> > > > > > address@hidden: int
> > > > > > address@hidden: constant 0 (zero)
> > > > > > """
> > > > > > last_index = globals.restart.last_index
> > > > > > last_block =
> > globals.restart.last_block
> > > > > > try:
> > > > > > # Just spin our wheels
> > > > > >
> > iter_result =
> > > > > > tarblock_iter.next()
> > > > > > while iter_result:
> > > > > > if
> > (tarblock_iter.previous_index
> > > > ==
> > > > > > last_index):
> > > > > > # If both the previous index and this
> > index
> > > > are
> > > > > > done, exit now
> > > > > > # before we hit the next index, to
> > prevent
> > > > > > skipping its first
> > > > > > # block.
> > > > > > if not
> > last_block
> > > > and
> > > > > > not
> > > > > > tarblock_iter.previous_block:
> > > > > > break
> > > > > > # Only
> > check
> > > > > > block number if
> > > > > > last_block is also a number
> > > > > > if last_block and
> > > > tarblock_iter.previous_block >
> > > > > > last_block:
> > > > > > break
> > > > > > if
> > > > > > tarblock_iter.previous_index
> > > > > > > last_index:
> > > > > > log.Warn(_("File %s complete in backup
> > > > set.\n"
> > > > > >
> > > > "Continuing
> > > > > > restart
> > > > > > on file %s.") %
> > > > > >
> > > > > > Kenneth Loafman <address@hidden> wrote:
> > > > > >
> > > > > > > It sounds like the backup restarted completely, rather than at
> > the
> > > > last
> > > > > > > known spot. What do the first 50 or so lines of the log contain?
> > > > > > >
> > > > > > > ...Ken
> > > > > > >
> > > > > > >
> > > > > > > On Tue, May 10, 2016 at 10:48 AM, <address@hidden> wrote:
> > > > > > >
> > > > > > > > But the question is why does duplicity need to do that at all?
> > > > > > > > Couldn't it go through the last hundred or so or some
> > reasonable
> > > > > > number?
> > > > > > > > Also, what about the original error which caused the restart
> > in the
> > > > > > > > first place? What is up with that that?
> > > > > > > >
> > > > > > > >
> > > > > > > > Kenneth Loafman <address@hidden> wrote:
> > > > > > > >
> > > > > > > > > No, there is no way to get rid of the loop, whichever one
> > you're
> > > > > > talking
> > > > > > > > > about, there are a lot of them.
> > > > > > > > >
> > > > > > > > > The main suggestion I have for anyone with multi-hour
> > backups is
> > > > to:
> > > > > > > > >
> > > > > > > > > 1. Use a local filesystem or USB drive for backup,
> > > > > > > > > 2. Transfer the backup using rsync or similar tool.
> > > > > > > > >
> > > > > > > > > The second suggestion I have is to chunk the backup into
> > > > reasonably
> > > > > > sized
> > > > > > > > > portions, 200-500GB each.
> > > > > > > > >
> > > > > > > > > ...Ken
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, May 10, 2016 at 7:02 AM, <address@hidden>
> > wrote:
> > > > > > > > >
> > > > > > > > > > Hi. I had to restart my full backup and its going through
> > > > every
> > > > > > file
> > > > > > > > it
> > > > > > > > > > had and it is taking 18 hours and its not done yet. I see
> > a
> > > > loop
> > > > > > in
> > > > > > > > > > there and I wonder if there is any way to shorten the time
> > or
> > > > get
> > > > > > rid
> > > > > > > > of
> > > > > > > > > > the loop altoggether?
> > > > > > > > > >
> > > > > > > > > > Thanks in advance for any suggestions.
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > Your life is like a penny. You're going to lose it. The
> > > > question
> > > > > > is:
> > > > > > > > > > How do
> > > > > > > > > > you spend it?
> > > > > > > > > >
> > > > > > > > > > John Covici
> > > > > > > > > > address@hidden
> > > > > > > > > >
> > > > > > > > > > _______________________________________________
> > > > > > > > > > Duplicity-talk mailing list
> > > > > > > > > > address@hidden
> > > > > > > > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > ----------------------------------------------------
> > > > > > > > > Alternatives:
> > > > > > > > >
> > > > > > > > > ----------------------------------------------------
> > > > > > > > > _______________________________________________
> > > > > > > > > Duplicity-talk mailing list
> > > > > > > > > address@hidden
> > > > > > > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > > > > > >
> > > > > > > > --
> > > > > > > > Your life is like a penny. You're going to lose it. The
> > question
> > > > is:
> > > > > > > > How do
> > > > > > > > you spend it?
> > > > > > > >
> > > > > > > > John Covici
> > > > > > > > address@hidden
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Duplicity-talk mailing list
> > > > > > > > address@hidden
> > > > > > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > > > > > >
> > > > > > >
> > > > > > > ----------------------------------------------------
> > > > > > > Alternatives:
> > > > > > >
> > > > > > > ----------------------------------------------------
> > > > > > > _______________________________________________
> > > > > > > Duplicity-talk mailing list
> > > > > > > address@hidden
> > > > > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > > > >
> > > > > > --
> > > > > > Your life is like a penny. You're going to lose it. The question
> > is:
> > > > > > How do
> > > > > > you spend it?
> > > > > >
> > > > > > John Covici
> > > > > > address@hidden
> > > > > >
> > > > > > _______________________________________________
> > > > > > Duplicity-talk mailing list
> > > > > > address@hidden
> > > > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > > > >
> > > > >
> > > > > ----------------------------------------------------
> > > > > Alternatives:
> > > > >
> > > > > ----------------------------------------------------
> > > > > _______________________________________________
> > > > > Duplicity-talk mailing list
> > > > > address@hidden
> > > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > >
> > > > --
> > > > Your life is like a penny. You're going to lose it. The question is:
> > > > How do
> > > > you spend it?
> > > >
> > > > John Covici
> > > > address@hidden
> > > >
> > > > _______________________________________________
> > > > Duplicity-talk mailing list
> > > > address@hidden
> > > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > >
> > >
> > > ----------------------------------------------------
> > > Alternatives:
> > >
> > > ----------------------------------------------------
> > > _______________________________________________
> > > Duplicity-talk mailing list
> > > address@hidden
> > > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
> > --
> > Your life is like a penny. You're going to lose it. The question is:
> > How do
> > you spend it?
> >
> > John Covici
> > address@hidden
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> > https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
>
> ----------------------------------------------------
> Alternatives:
>
> ----------------------------------------------------
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?
John Covici
address@hidden
_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk