duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] Out of space error while restoring a file


From: Laurynas Biveinis
Subject: Re: [Duplicity-talk] Out of space error while restoring a file
Date: Sun, 7 Oct 2012 10:03:40 +0300

2012/9/30  <address@hidden>:
> On 29.09.2012 09:50, Laurynas Biveinis wrote:
>> 2012/9/26  <address@hidden>:
>>> On 26.09.2012 11:42, Laurynas Biveinis wrote:
>>> [SNIP]
>>>>>>
>>>>>>>> So, "twice the size of the restored file + one volume?" But originally
>>>>>>>> it failed with 80GB free for a 30GB file, so it's even more than that.
>>>>>>>
>>>>>>> right, we would have to disect filesystem usage during restoring to 
>>>>>>> find out what is going on here. actually the v9 dump tells you when 
>>>>>>> temp files are created and deleted (though not their size) so you can 
>>>>>>> work through that if you want.
>>>>>>>
>>>>>>> alternatively you have to observe what happens in the tmp folder during 
>>>>>>> restore ;)
>>>>>>
>>>>>> Here's the open file snapshot around volume 704 (out of ~750):
>>>>>> http://pastebin.com/pGFH6Wju
>>>>>>
>>>>>> Note that "SandÄ—lys" has been replaced with "Elements", that's another
>>>>>> volume (so that locally-downloaded backup sets would fit)
>>>>>>
>>>>>> Thanks again,
>>>>>>
>>>>>
>>>>> ok, just wanted to check the command line output promised above, just to 
>>>>> see there was nothing attached ;)
>>>>
>>>> Oops. Fixed.
>>>>
>>>>> wrt to the lsof output. it shows
>>>>> duplicity 8946 laurynas    7u   REG   8,33 25211305984     8622 
>>>>> /media/Elements/Laurynas/tmp/duplicity-DL3ypc-tempdir/tmpgxn0pY (deleted)
>>>>> and
>>>>> duplicity 8946 laurynas   12u   REG   8,33 34465812480     8614 
>>>>> /media/Elements/Laurynas/tmp/duplicity-DL3ypc-tempdir/tmpYZgzig (deleted)
>>>>>
>>>>> and some others, all of them have the tag (deleted), so i assume the 
>>>>> space was already freed again?
>>>>
>>>> Nope, it's taken space.
>>>>
>>>>> even if not this is what we assumed: 2 times the 30GB file plus some.
>>>>>
>>>>> what would be interesting is a "multiplexed" output of 'duplicity -v9' 
>>>>> _and_ and a periodic 'ls -la' of TMP . so one could see what files 
>>>>> duplicity creates/deletes and how this reflects in the TMP folder. i 
>>>>> believe this would be possible with bash magic.
>>>>
>>>> I'l see about producing this. But, ls -la will not show those big
>>>> deleted files. I will try to include lsof as well, perhaps filtered to
>>>> remove open net sockets and the like.
>>>>
>>>
>>> first off, the v9 output does not show any tmp files named like tmpYZgzig 
>>> but only mktemp-MRoS1d-741 ones. so i assume the default python tempfile 
>>> does not inform us
>>>
>>> or
>>>
>>> you are not using the latest patch?
>>> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/revision/860
>>
>> I am using the latest patch but not the latest duplicity code: I
>> patched Ubuntu-shipped 0.6.18. Is that a good version to test or do
>> you want me to use the latest trunk or any other version?
>
> should be fine for the issue at hand
>
>>> regarding the used space. did you monitor via du/ls/df the duplicity tmp 
>>> folder and can confirm that it uses up much more than the 2xbiggest file 
>>> plus some? how much?
>>
>> Will do.
>>
>
> please.. and come back with results. ede/duply.net

OK it took a while, but here it is.

The script I used

duplicity -v9 -t19D --file-to-restore file &
duplicity_pid=$!
lsof -p $duplicity_pid +r 300 &

while ps aux | grep [d]uplicity >/dev/null 2>&1
do
    date
    ls -lAR $TMP
    du -c $TMP
    df -h --sync $TMP
    sleep 300s
done

-- 
Laurynas

Attachment: restore.log.bz2
Description: BZip2 compressed data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]