duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] webdavs and large sync volume failing


From: address@hidden
Subject: Re: [Duplicity-talk] webdavs and large sync volume failing
Date: Sat, 6 Feb 2016 09:09:12 +0100

Hi,

I have reproduced the error with verbosity 9 and sent the logfiles to 
ede/duply.net.
As stated in that mail it looks like this is a memory issue (/var/log/messages 
shows that oom-killer kicks in)

In parallel I have now upgraded my system to 
duplicity 0.7.06
duply 1.11.1

and re-run the backup with backend lftp+webdavs as proposed below. This works 
fine without errors

but

If I check the status of this backup via the same backend lftp+webdavs it shows 
no backup present.
If I change back to webdavs backend for the status command I can see the 1 
successfull full backup I created with the lftp+webdavs backend. 
I tested the same with a small backup repository, here the backend lftp+webdavs 
works fine both for the backup and the status command

A quick investigation I did shows that the status command for the big backup 
(roughly 14GB of photos) has an issue when retrieving the file info from the 
webdav server. There is no data returned - in the log below there should be a 
long file list after STDOUT: but this is empty.

——— snip from log —— 
CMD: lftp -c 'source 
'/home/backup/tmp/duplicity-uzDqze-tempdir/mkstemp-yXoZe9-1'; cd 
'backup/photo/1971-2005/' || exit 0; ls'
Reading results of 'lftp -c 'source 
'/home/backup/tmp/duplicity-uzDqze-tempdir/mkstemp-yXoZe9-1'; cd 
'backup/photo/1971-2005/' || exit 0; ls''
STDERR:
---- Resolving host address...
---- 1 address found: xxxxxxx
---- Connecting to sd2dav. xxxxxxx port 443
---- Sending request...
---> PROPFIND / HTTP/1.1
---> Host: sd2dav. xxxxxxx
---> User-Agent: lftp/4.6.0
---> Accept: */*
---> Depth: 0
---> Authorization: Basic xxxxxxx
---> Connection: keep-alive
---> 
Certificate: xxxxxxx
  Trusted
<--- HTTP/1.1 207 Multi-Status
<--- Date: Sat, 06 Feb 2016 06:55:09 GMT
<--- Server: Apache
<--- ETag: "xxxxxxx ="
<--- Content-Length: 1857
<--- Content-Type: text/xml; charset="utf-8"
<--- Vary: Accept-Encoding
<--- Keep-Alive: timeout=3, max=100
<--- Connection: Keep-Alive
<--- 
---- Receiving body...
---- Hit EOF
---- Closing HTTP connection
---- Connecting to sd2dav. xxxxxxx port 443
---- Sending request...
---> PROPFIND /backup/photo/1971-2005/ HTTP/1.1
---> Host: sd2dav. xxxxxxx
---> User-Agent: lftp/4.6.0
---> Accept: */*
---> Depth: 0
---> Authorization: Basic xxxxxxx
---> Connection: keep-alive
---> 
Certificate: xxxxxxx
  Trusted
<--- HTTP/1.1 207 Multi-Status
<--- Date: Sat, 06 Feb 2016 06:55:19 GMT
<--- Server: Apache
<--- ETag: "xxxxxxx ="
<--- Content-Length: 1368
<--- Content-Type: text/xml; charset="utf-8"
<--- Vary: Accept-Encoding
<--- Keep-Alive: timeout=3, max=100
<--- Connection: Keep-Alive
<--- 
---- Receiving body...
---- Hit EOF
---- Closing HTTP connection
---- Connecting to sd2dav. xxxxxxx port 443
---- Sending request...
---> GET /backup/photo/1971-2005/ HTTP/1.1
---> Host: sd2dav. xxxxxxx
---> User-Agent: lftp/4.6.0
---> Accept: */*
---> Authorization: Basic xxxxxxx
---> Connection: keep-alive
---> 
Certificate: xxxxxxx
  Trusted
<--- HTTP/1.1 200 OK
<--- Date: Sat, 06 Feb 2016 06:55:20 GMT
<--- Server: Apache
<--- Last-Modified: Mon, 01 Feb 2016 19:44:00 GMT
<--- ETag: "xxxxxxx ="
<--- Accept-Ranges: bytes
<--- Content-Length: 0
<--- Content-Type: text/html; charset="UTF-8"
<--- Vary: Accept-Encoding
<--- Keep-Alive: timeout=3, max=100
<--- Connection: Keep-Alive
<--- 
---- Receiving body...
---- Received all
---- Closing HTTP connection

STDOUT:

Local and Remote metadata are synchronized, no sync needed.

——— snip end from log ——  

> Am 02.02.2016 um 21:27 schrieb address@hidden:
> 
> On 02.02.2016 21:18, address@hidden wrote:
>> Hi,
>> 
>> I am trying to use Duplicity and duply to backup appr 60GB data to webdavs 
>> storage.
>> 
>> The script is running on a raspberrypi with raspbian jessie
>> 
>> Small backups are working.
>> I have tried to run this backup in one go, it failed; I then split into 15GB 
>> chunks it failed again with exit code 137
>> 
>> Any ideas what to do / where to look for?
> 
> can you run duplicity in max. verbosity '-v9' and send me the _complete_ 
> output privately?
> 
>> Is the webdavs access not stable enough for such backup?
> 
> should make no difference. the backup is split into volumes of the same size 
> anyway.
> 
>> Is it maybe a performance issue with the raspberry
>> 
>> btw I started the backup script via a cron job, it was then running several 
>> hours before it failed.
>> 
> 
> what's your duplicity version? with the latest duplicity you can install lftp 
> and use that as alernative webdav backend via lftp+webdav://
> 
> ..ede/duply.net
> 
> 




reply via email to

[Prev in Thread] Current Thread [Next in Thread]