[Duplicity-talk] --no-compression is broken WAS: Backup not restorable with no compression?

[edgar . soldin]

i spent some time on debugging this. here is the result so far

renaming the volumes works because they are actually gzipped. this is because 
we have two bugs

1. in duplicity.write_multivol() we always create GZIPFiles, around line 396
 
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/bin/duplicity#L396

2. our tarfile implementation does not complain when it can't read the file 
properly, but seems to deliver zero entries instead. i didn't dig up where it 
should do that but expect somewhere in 'tarfile.py'.

sure, we could easily remove the option. it seems to be not even documented in 
the man page.
BUT i strongly feel we should keep and fix it as 
- it'll help others to understand our data format better and 
- maybe in the future will make it easier to implement an even better 
replacement data format.

unfortunately my python isn't good enough to implement the PlainWriteFile. i 
assume one could reuse the code of GzipWriteFile here
 
http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/duplicity/gpg.py#L347
and write into a python file object instead.

any hints? Ken, Mike?  

bug number 2. is not that serious, but leaves a stale taste, as a proper error 
would be expected.

..ede/duply.net


On 27.01.2014 18:03, Knut Krause wrote:
> OK, I'd have to dig into that. Maybe I will. Until then it would be at least 
> great if the option is disabled. This is just 1 minute work for the devs and 
> it would prevent duplicity from creating a backup it cannot restore any more.
> 
> Knut
> 
> Am Montag, 27. Januar 2014, 17:39:18 schrieb address@hidden:
>> good to know! maybe some warm heart comes around and does just that. the
>> reason that no compression isn't fixed is simply - no one cares enough for
>> this option, obviously.
>>
>> speaking of "you guys should". how about you dig in and supply a patch?
>> python is not brain surgery ;)
>>
>> the lesson for you should be. don't trust, verify! while duplicity is quite
>> robust, it still has nooks and corners. just now, 0.6.23 fixed a bug that
>> resulted in data loss when a backup was resumed.
>>
>> all this can be easily circumvented by periodically verifying your backups.
>>
>> ..ede/duply.net
>>
>> On 27.01.2014 16:06, Knut Krause wrote:
>>> Just for your information: renaming the *.difftar files to *.difftar.gz
>>> seems to result in a working restore again.
>>>
>>> You guys should really either remove or FIX the --no-compression option
>>> though. This is behaviour is not acceptable for backup software.
>>>
>>>
>>> Knut
>>>
>>> Am Montag, 27. Januar 2014, 15:21:14 schrieb Knut Krause:
>>>> Hi,
>>>>
>>>> I created a backup using --no-compression. The filenames are
>>>> duplicity-inc.20140113T152430Z.to.20140123T112751Z.vol8.difftar
>>>> without the .gz ending.
>>>>
>>>> file on the other hand says:
>>>> duplicity-inc.20140113T152430Z.to.20140123T112751Z.vol9.difftar:
>>>> gzip compressed data, last modified: Thu Jan 23 13:16:42 2014, max
>>>> compression
>>>>
>>>> Now I found
>>>> http://lists.gnu.org/archive/html/duplicity-talk/2013-03/msg00001.html
>>>> and
>>>> https://bugs.launchpad.net/duplicity/+bug/1029516
>>>> which suggests that this option is buggy since 2012.
>>>>
>>>> Any suggestions how to fix this? Can I just rename the files to *.gz or
>>>> will this break the signature stuff?
>>>>
>>>>
>>>> Knut
>>>>
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> address@hidden
>>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk
>