Re: [rdiff-backup-users] Re: Remote encrypted backup with slow connection.

[Piotr Karbowski]

On Tue, Sep 22, 2009 at 10:51 AM, Dominic Raferd <address@hidden> wrote:
> Piotr Karbowski wrote:
>>
>> On Mon, Sep 21, 2009 at 8:21 PM, Dominic Raferd <address@hidden>
>> wrote:
>>
>>>
>>> Piotr Karbowski wrote:
>>>
>>>>
>>>> On Mon, Sep 21, 2009 at 2:02 PM, Matthew Miller <address@hidden>
>>>> wrote:
>>>>
>>>>
>>>>>
>>>>> On Mon, Sep 21, 2009 at 01:58:11PM +0200, Piotr Karbowski wrote:
>>>>>
>>>>>
>>>>>>
>>>>>> local rdiff-backup dir with remote server but how? If I will use for
>>>>>> example rsync it still need to check whole files for changes (read,
>>>>>> download it) and upload only new. I hope you will understand what I
>>>>>> need and help me.
>>>>>>
>>>>>>
>>>>>
>>>>> rsync won't check whole files unless you give the -c flag. Otherwise,
>>>>> it
>>>>> just compares metadata. I don't know if that's also the case with
>>>>> rdiff-backup, but I assume so.
>>>>>
>>>>>
>>>>
>>>> So I need to know how rdiff default compares data, if by size and
>>>> mod-time, it will not be so painful but still itefficient  will download
>>>> changed
>>>> files to generate diff.
>>>>
>>>
>>> Rdiff-backup is designed to be ultra-efficient at this activity. It only
>>> sends the changes in a file over the wire, not the whole file. To do this
>>> it
>>> uses the librsync library which is effectively the same as rsync. You can
>>> read more about the technique at http://en.wikipedia.org/wiki/Rsync.
>>> rdiff-backup does not use file times to determine whether to do backups.
>>> It
>>> can backup very large files with small changes very quickly.
>>>
>>> Dominic
>>>
>>>
>>
>> You dont understand me, rdiff-backup is efficient, but to make diff it
>> must read WHOLE file, on remote nfs or sshfs it is SLOOOOW and
>> painful
>
> Sorry I get it now. But I think rdiff-backup and rsync require a separate
> computer at the remote end in order to optimise transfers, so if you are
> just accessing a remote share using sshfs or similar then they can still
> work of course but as you realise they will be slow. I guess it is not
> possible for you to run rdiff-backup (or rsync) at the remote end as well?
>
> You could run rdiff-backup locally to create a backup store and then mirror
> this store to the remote share using rcp. Still it will be slow because
> rdiff-backup always stores the latest copy of each file in full and so if
> this changes even slightly then the whole file will must be transferred by
> rcp.
>
> Duplicity http://duplicity.nongnu.org/ might work better for you, because it
> uses forward diffs. Also its archives are secure.
>
> Although not directly relevant I found a page here
> http://www.psc.edu/networking/projects/hpn-ssh/ which provides a patch to
> greatly speed up OpenSSH in some situations.

Duplicity is interesing project. What you think about using
rdiff-backup to create local backup, for example in /backups and then
send this /backups to remote server by duplicity? As far as I know
duplicity is encrypted so I DONT need using encfs, dmcrypt or other -
only ssh access is needed (realy I dont need duplicity on remote
server?).

I just want be able to send _ENCRYPTED_ backups to remote server where
I have only ssh access (sftp/scp work).