Re: [Duplicity-talk] Amazon S3/socket error problems

[Mark Wardle]

Duplicity with S3 has never worked on my machine.

I would be very happy to run a test program if it helps!

Mark

On 05/01/2008, Mitchell Garnaat <address@hidden> wrote:
> There definitely isn't any inherent problem with boto/python/s3 and the Mac
> platform because I developed boto on a Mac and use it everyday.  Does it
> fail in this way consistently?  Has it ever worked?  Would you be willing to
> run a small Python test program if I mailed it to you?
>
> Mitch
>
>
> On Jan 5, 2008 1:44 PM, Mark Wardle <address@hidden> wrote:
> > Dear Richard,
> >
> > Yes - I have the boto package installed.
> >
> > It was installed via macports, but just in case I have manually
> > installed it from the .tar.gz file as well.
> >
> > Thanks,
> >
> > Mark
> >
> >
> >
> >
> >
> >
> >
> > On 05/01/2008, Richard Scott <address@hidden> wrote:
> > > Hi,
> > >
> > > Do you have the Python Boto package installed?
> > >
> > > http://code.google.com/p/boto/
> > >
> > > Duplicity needs this to talk to Amazon S3.
> > >
> > > Cheers,
> > >
> > > Rich.
> > >
> > > > Dear all,
> > > >
> > > >
> > > > I have been using duplicity successfully to backup (via scp/ssh) and I
> > > > must say it is great - so thank you!
> > > >
> > > > However, I have started to look at Amazon S3. I have been using r3sync
> > > > and jungledisk successfully, but have not had any success with
> duplicity.
> > > >
> > > > My system is MacOSX, 10.4.11, with a current macports installation. In
> > > > addition, I have upgraded the version of duplicity (manually) to the
> latest version as I had these
> > > > same problems with the previous version.
> > > >
> > > > duplicity @0.4.9_0 (active) librsync @0.9.7_1 (active) py-boto @1.0a_0
> (active) py-pexpect @2.1_0
> > > > (active)
> > > > py-gnupg @ 0.3.2_1 (active) py25-boto @1.0a_0 (active) python24
> @2.4.4_1+darwin_8 (active) python25
> > > > @2.5.1_4+darwin_8 (active)
> > > >
> > > >
> > > > I repeatedly get socket errors. I have reviewed the mailing list, and
> > > > have seen the prior thread regarding this issue, but did not see a
> solution posted.
> > > >
> > > > I have tested this using a cable modem internet connection as well as
> > > > a university network (T1) with the same error. In addition, I have no
> problem with other s3
> > > > clients.
> > > >
> > > > I have not tried using boto directly as I don't know python but wonder
> > > > whether this a specific Mac OS X problem as other users haven't
> reported it?
> > > >
> > > > The transcript is shown below.
> > > >
> > > >
> > > > Are there any workarounds possible, or can anyone suggest any
> > > > tests/scripts to run to try and isolate the problem a little more?
> > > >
> > > > Alternatively, I could run a hybrid solution using duplicity to create
> > > > an archive and then use s3sync to upload the files to S3 separately. I
> presume there is no
> > > > complication in moving an archive around in this way?
> > > >
> > > > Many thanks,
> > > >
> > > >
> > > > Best wishes,
> > > >
> > > >
> > > > Mark
> > > >
> > > >
> > > > ================================================
> > > >
> > > >
> > > >
> > > > Daisy:~ mark$ duplicity -v9 /Users/mark/Repos s3+http://wardle
> > > > Main action: inc
> > > > Collection Status
> > > > -----------------
> > > > Connecting with backend: BotoBackend
> > > > Archive dir: None
> > > >
> > > >
> > > > Found 0 backup chains without signatures.
> > > > No backup chains with active signatures found
> > > > No orphaned or incomplete backup sets found.
> > > > Last full backup date: Thu Jan  1 01:00:00 1970
> > > > No signatures found, switching to full backup.
> > > > Using temporary directory /tmp/duplicity-YkKddu-tempdir
> > > > Registering (mktemp) temporary file
> > > > /tmp/duplicity-YkKddu-tempdir/mktemp-Kzk1h6-1
> > > > Registering (mktemp) temporary file
> > > > /tmp/duplicity-YkKddu-tempdir/mktemp-Xb73NX-2
> > > > Selecting /Users/mark/Repos
> > > > Comparing () and None
> > > > Getting delta of (() /Users/mark/Repos dir) and None
> > > > Generating delta - new file: .
> > > > Selecting /Users/mark/Repos/.DS_Store
> > > > Comparing ('.DS_Store',) and None
> > > > Getting delta of (('.DS_Store',) /Users/mark/Repos/.DS_Store reg) and
> None
> > > > Generating delta - new file: .DS_Store
> > > > Selecting /Users/mark/Repos/Papers
> > > > Comparing ('Papers',) and None
> > > > Getting delta of (('Papers',) /Users/mark/Repos/Papers dir) and None
> > > > Generating delta - new file: Papers
> > > > Selecting /Users/mark/Repos/Papers/README.txt
> > > > Comparing ('Papers', 'README.txt') and None
> > > > Getting delta of (('Papers', ' README.txt')
> > > > /Users/mark/Repos/Papers/README.txt reg) and None
> > > > Generating delta - new file: Papers/README.txt
> > > > Selecting /Users/mark/Repos/Papers/conf
> > > > Comparing ('Papers', 'conf') and None
> > > > Getting delta of (('Papers', 'conf') /Users/mark/Repos/Papers/conf
> dir) and None
> > > > Generating delta - new file: Papers/conf
> > > > Selecting /Users/mark/Repos/Papers/conf/authz
> > > > Comparing ('Papers', 'conf', 'authz') and None
> > > > Getting delta of (('Papers', 'conf', 'authz')
> > > > /Users/mark/Repos/Papers/conf/authz reg) and None
> > > > Generating delta - new file: Papers/conf/authz
> > > > Selecting /Users/mark/Repos/Papers/conf/passwd
> > > > Comparing ('Papers', 'conf', 'passwd') and None
> > > > Getting delta of (('Papers', 'conf', 'passwd')
> > > > /Users/mark/Repos/Papers/conf/passwd reg) and None
> > > > Generating delta - new file: Papers/conf/passwd
> > > > Selecting /Users/mark/Repos/Papers/conf/svnserve.conf
> > > > Comparing ('Papers', 'conf', ' svnserve.conf') and None
> > > > Getting delta of (('Papers', 'conf', 'svnserve.conf')
> > > > /Users/mark/Repos/Papers/conf/svnserve.conf reg) and
> None
> > > > Generating delta - new file: Papers/conf/svnserve.conf
> > > > Selecting /Users/mark/Repos/Papers/dav
> > > > Comparing ('Papers', 'dav') and None
> > > > Getting delta of (('Papers', 'dav') /Users/mark/Repos/Papers/dav dir)
> and None
> > > >
> > > >
> > > > [...] cut
> > > > Generating delta - new file: Papers/db/revs/0
> > > > Selecting /Users/mark/Repos/Papers/db/revs/1
> > > > Comparing ('Papers', 'db', 'revs', '1') and None
> > > > Getting delta of (('Papers', 'db', 'revs', '1')
> > > > /Users/mark/Repos/Papers/db/revs/1 reg) and None
> > > > Generating delta - new file: Papers/db/revs/1
> > > > Uploading s3+http://wardle/duplicity-
> full.2008-01-05T11:19:48Z.vol1.difftar.gpg
> > > > Caught a socket error, trying to recover
> > > > Caught a socket error, trying to recover
> > > > [...] cut   (endlessly loops until I CTRL-C - which interestingly
> > > > always interrupts gpg rather than duplicity directly?) Caught a socket
> error, trying to recover
> > > > ^CUpload
> 's3+http://wardle/duplicity-full.2008-01-05T11:19:48Z.vol1.difftar.gpg'
> > > > failed (attempt #1)
> > > >
> > > > gpg: Interrupt caught ... exiting
> > > >
> > > >
> > > >
> > > > --
> > > > Dr. Mark Wardle
> > > > Specialist registrar, Neurology
> > > > Cardiff, UK
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Duplicity-talk mailing list
> > > > address@hidden
> > > >
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Duplicity-talk mailing list
> > > address@hidden
> > >
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > >
> > >
> > >
> ______________________________________________________________________
> > > This email has been scanned by the MessageLabs Email Security System.
> > > For more information please visit
> http://www.messagelabs.com/email
> > >
> ______________________________________________________________________
> > >
> >
> >
> > --
> >
> >
> >
> > Dr. Mark Wardle
> > Specialist registrar, Neurology
> > Cardiff, UK
> >
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> >
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >
>
>
> ______________________________________________________________________
>  This email has been scanned by the MessageLabs Email Security System.
>  For more information please visit
> http://www.messagelabs.com/email
> ______________________________________________________________________
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>
>


-- 
Dr. Mark Wardle
Specialist registrar, Neurology
Cardiff, UK