[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Duplicity-talk] Having to specify incrementals
From: |
edgar . soldin |
Subject: |
Re: [Duplicity-talk] Having to specify incrementals |
Date: |
Tue, 16 Apr 2013 17:58:05 +0200 |
User-agent: |
Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130328 Thunderbird/17.0.5 |
On 16.04.2013 17:54, Elvar wrote:
>
> On 4/4/2013 11:49 AM, address@hidden wrote:
>> On 04.04.2013 18:10, Elvar wrote:
>>> On 4/4/2013 11:05 AM, address@hidden wrote:
>>>> On 04.04.2013 18:02, Elvar wrote:
>>>>> On 4/4/2013 10:24 AM, address@hidden wrote:
>>>>>> On 04.04.2013 17:16, Elvar wrote:
>>>>>>> On 3/29/2013 5:29 PM, address@hidden wrote:
>>>>>>>> On 29.03.2013 22:24, Elvar wrote:
>>>>>>>>> On 3/29/2013 1:59 PM, address@hidden wrote:
>>>>>>>>>> On 29.03.2013 19:35, Elvar wrote:
>>>>>>>>>>> On 3/29/2013 11:49 AM, address@hidden wrote:
>>>>>>>>>>>> On 29.03.2013 17:31, Elvar wrote:
>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I recently started using Duplicity to perform offsite backups for
>>>>>>>>>>>>> a linux server of ours. When I performed a simulated disaster
>>>>>>>>>>>>> recovery scenario I found that the only data I had been able
>>>>>>>>>>>>> restore was data from the initial full backup. It doesn't appear
>>>>>>>>>>>>> that Duplicity was automatically doing incrementals despite the
>>>>>>>>>>>>> data growing. Below is the command I'm using...
>>>>>>>>>>>>>
>>>>>>>>>>>>> FTP_PASSWORD='somepass' PASSPHRASE='somepass' duplicity /mnt
>>>>>>>>>>>>> ftps://address@hidden
>>>>>>>>>>>>>
>>>>>>>>>>>>> Shouldn't that automatically assume incrementals if the full had
>>>>>>>>>>>>> already been done?
>>>>>>>>>>>>>
>>>>>>>>>>>> yes. what's the output of collection-status?
>>>>>>>>>>>>
>>>>>>>>>>>> ..ede/duply.net
>>>>>>>>>>> After having manually ran an incremental or two, here is the
>>>>>>>>>>> current status.
>>>>>>>>>>>
>>>>>>>>>>> Import of duplicity.backends.giobackend Failed: No module named gio
>>>>>>>>>>> Import of duplicity.backends.sshbackend Failed: No module named
>>>>>>>>>>> paramiko
>>>>>>>>>>> LFTP version is 4.3.3
>>>>>>>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>>>>>>>> Last full backup date: Thu Mar 28 17:03:26 2013
>>>>>>>>>>> Collection Status
>>>>>>>>>>> -----------------
>>>>>>>>>>> Connecting with backend: FTPSBackend
>>>>>>>>>>> Archive dir: /root/.cache/duplicity/cb471964ea71f51bdbff729d2a8e763e
>>>>>>>>>>>
>>>>>>>>>>> Found 0 secondary backup chains.
>>>>>>>>>>>
>>>>>>>>>>> Found primary backup chain with matching signature chain:
>>>>>>>>>>> -------------------------
>>>>>>>>>>> Chain start time: Thu Mar 28 17:03:26 2013
>>>>>>>>>>> Chain end time: Fri Mar 29 11:23:11 2013
>>>>>>>>>>> Number of contained backup sets: 6
>>>>>>>>>>> Total number of contained volumes: 61
>>>>>>>>>>> Type of backup set: Time: Num
>>>>>>>>>>> volumes:
>>>>>>>>>>> Full Thu Mar 28 17:03:26 2013 50
>>>>>>>>>>> Incremental Thu Mar 28 19:24:47 2013
>>>>>>>>>>> 1
>>>>>>>>>>> Incremental Fri Mar 29 09:23:59 2013
>>>>>>>>>>> 1
>>>>>>>>>>> Incremental Fri Mar 29 09:33:08 2013
>>>>>>>>>>> 1
>>>>>>>>>>> Incremental Fri Mar 29 10:12:58 2013
>>>>>>>>>>> 1
>>>>>>>>>>> Incremental Fri Mar 29 11:23:11 2013
>>>>>>>>>>> 7
>>>>>>>>>>> -------------------------
>>>>>>>>>>> No orphaned or incomplete backup sets found.
>>>>>>>>>>>
>>>>>>>>>> dunno what your complaining about. it clearly states
>>>>>>>>>>
>>>>>>>>>> 1 x full
>>>>>>>>>> 5 x incrementals
>>>>>>>>>>
>>>>>>>>>> run again without forced incremental and see if it adds full or incr.
>>>>>>>>>>
>>>>>>>>>> ..ede
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> So I just ran it again and I see it says "NewFiles 0" and
>>>>>>>>> "NewFileSize 0". The content I'm backing up is an email archive that
>>>>>>>>> uses Maildir format for storage. I know for certain several hundred
>>>>>>>>> emails or more have landed in this archive since my last incremental
>>>>>>>>> yet this last job doesn't seem to see it. So, my question is, why
>>>>>>>>> isn't the incremental job grabbing the latest data?
>>>>>>>>>
>>>>>>>>> Import of duplicity.backends.giobackend Failed: No module named gio
>>>>>>>>> Import of duplicity.backends.sshbackend Failed: No module named
>>>>>>>>> paramiko
>>>>>>>>> LFTP version is 4.3.3
>>>>>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>>>>>> Last full backup date: Thu Mar 28 17:03:26 2013
>>>>>>>>> --------------[ Backup Statistics ]--------------
>>>>>>>>> StartTime 1364591400.73 (Fri Mar 29 16:10:00 2013)
>>>>>>>>> EndTime 1364591418.34 (Fri Mar 29 16:10:18 2013)
>>>>>>>>> ElapsedTime 17.62 (17.62 seconds)
>>>>>>>>> SourceFiles 29069
>>>>>>>>> SourceFileSize 2409332518 (2.24 GB)
>>>>>>>>> NewFiles 0
>>>>>>>>> NewFileSize 0 (0 bytes)
>>>>>>>>> DeletedFiles 0
>>>>>>>>> ChangedFiles 0
>>>>>>>>> ChangedFileSize 0 (0 bytes)
>>>>>>>>> ChangedDeltaSize 0 (0 bytes)
>>>>>>>>> DeltaEntries 0
>>>>>>>>> RawDeltaSize 0 (0 bytes)
>>>>>>>>> TotalDestinationSizeChange 103 (103 bytes)
>>>>>>>>> Errors 0
>>>>>>>>> -------------------------------------------------
>>>>>>>>>
>>>>>>>> what happened to your first issue? is that solved?
>>>>>>>>
>>>>>>>> wrt. 0byte change . this can happen when software does not update
>>>>>>>> timestamps correctly. but i doubt that's the case here.
>>>>>>>> how about simply restoring the latest archive and binary compare it to
>>>>>>>> the current state? after that run 'verify' and see if that says there
>>>>>>>> is one.
>>>>>>>>
>>>>>>>> ..ede/duply.net
>>>>>>>>
>>>>>>>>
>>>>>>> Ok, I think I may have figured out the issue. When I was running the
>>>>>>> Duplicity backups initially I was doing it on a read only iscsi mounted
>>>>>>> volume. I noticed if I mounted without read-only I didn't have the same
>>>>>>> issue with it not backing up newer data. Is this expected behavior?
>>>>>>>
>>>>>> no, please check if timestamps are correctly changing when the iscsi
>>>>>> device is mounted read only.
>>>>>>
>>>>>> ede/duply.net
>>>>>>
>>>>>>
>>>>> The timestamps of the data I'm backing up? Well, the data is almost
>>>>> entirely archived email being stored in Maildir format. There is
>>>>> constantly new files being created every few seconds. For whatever
>>>>> reason, when mounted read-only, it skips the new files and no data
>>>>> changes. If I run the same exact command mounting rw, it backs up all new
>>>>> files and modified files since the last backup.
>>>>>
>>>> duplicity checks if timestamps changed to determine the need for backup.
>>>> check for your maildir files if timestamps change over time when mounted
>>>> ro. probably they don't and that causes the problem?
>>>>
>>>> ..ede
>>>>
>>> The appliance which is doing the email archiving isn't mounted ro though so
>>> it should be updating the time & date stamps as it performs writes.
>>> Wouldn't a separate server mounting the same partition only in ro mode
>>> still see changes?
>>>
>>>
>>>
>> generally yes, the timestamps should be updated. but i suppose they are not.
>> please check manually if they are and report back.
>>
>> ..ede
>
> So I found that whether I mounted it read only or not didn't matter. The only
> way it was correctly seeing the updated time/date stamps was to remount the
> iscsi volume just before each backup job. I'm guessing this is because two
> separate linux hosts are mounting the same ext3 partition and it is not a
> clustered FS. Either way, by adding umount and mount commands to my script it
> is working fine now. I do have a different question but I'll start a new
> topic for that.
>
thanks for keeping us posted. wrt the iscsi issue. you should contact the
author or the software's mailing list. either it is a bug or there is something
to setup. a file system that does not update timestamps is missing half of it's
purpose ;)
.. ede/duply.net