Re: [Duplicity-talk] no-compression still compresses?

[edgar . soldin]

i assume you mean in gpg.GzipWriteFile()?

if not, changes to gpg.py, do not change the volumes if --no-encryption is 
enabled as well. check the man page
http://duplicity.nongnu.org/duplicity.1.html
additionally you can manipulate the gpg options via --gpg-options already, 
including defining something like "--compress-algo=bzip2 
--bzip2-compress-level=9"
no need to hack it in there.

not to rain on your parade, but everything is generally in place. there is just 
a bug with --no-compression. to conclude

default, results in gpg encrypted and compressed volumes 
--no-encryption, results in gzipped volumes
--no-compression --no-encryption, results in tar compatible volumes (or at 
least should)

i see you could be mistaken that --no-compression would have an effect on gzip 
or gpg, but it does not.

if you really want gzipped volumes which are effectively uncompressed you can 
patch gpg.GzipWriteFile() to honour a parameter --gzip-level .. see how it is 
implemented for --gpg-options. you'll need to modify 

gpg.py
globals.py
commandline.py
bin/duplicity.1 (manpage explanations)

for that.


..ede/duply.net



On 07.03.2013 14:07, Cory Coager wrote:
> Not sure if I was clear or not but I only made change to gpg.py to change 
> compression from 6 to 0.  The file was still gzipped yes, but there was no 
> compression.  The important part is, the backups and restores were all 
> successful.
> 
> If you are adamant about not using gzip when there is no compression this 
> will require significant more code changes.  Perhaps a better way is to 
> expose a flag to control compression level?
> 
> On Thu, Mar 7, 2013 at 4:51 AM, <address@hidden <mailto:address@hidden>> 
> wrote:
> 
>     no time to "handle" this .. sorry, even if i'd have other priorities as 
> --no-compression is nothing i use with duplicity.
> 
>     the switch essentially disable gzip alltogether, hence the resulting 
> fiules would be untarable. simply switching the compression level to zero 
> does not achieve that.
> 
>     ..ede/duply.net <http://duply.net>
> 
>     On 07.03.2013 02:21, Cory Coager wrote:
>     > I did some troubleshooting on this and here is what I came up with.
>     >
>     > I was able to force the output files to be tar files and not gzipped by 
> changing gpg.py:
>     > - gzip_file = gzip.GzipFile(None, "wb", 6, file_counted)
>     > + gzip_file = file_counted
>     >
>     > However, this still broke restores.  I believe it still wants gzip 
> files to be present so I guess a better way to handle this is changing the 
> compression to 0 which will still gzip the files but turn compression off.  
> This test was successful for both backup and restore.
>     >
>     > Not sure how you want to handle this but those are my thoughts.
>     >
>     > On 03/06/2013 03:10 PM, address@hidden <mailto:address@hidden> wrote:
>     >> in bin/duplicity probably here
>     >> 
> http://bazaar.launchpad.net/~duplicity-team/duplicity/0.6-series/view/head:/bin/duplicity#L383
>  
> <http://bazaar.launchpad.net/%7Eduplicity-team/duplicity/0.6-series/view/head:/bin/duplicity#L383>
>     >>
>     >> also you should research the code that was added when --no-compression 
> switch was added.. search changelog for a circa point in time and then look 
> in the repositories history for the commit that added it.
>     >> i have the feeling something was lost, removed erroneously somewhen, 
> as i recall it worked after the switch was included.
>     >>
>     >> ..ede/duply.net <http://duply.net>
>     >>
>     >> On 06.03.2013 20:59, Cory Coager wrote:
>     >>> If someone can point me in the right direction I can.  I tried 
> troubleshooting this yesterday by hacking the code and wasn't able to get it 
> working.  Where should I be looking?
>     >>>
>     >>> On Wed, Mar 6, 2013 at 2:21 PM, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>> 
> wrote:
>     >>>
>     >>>      thanks for following up. could we interest you in fixing this? 
> any contribution is valued lots.
>     >>>
>     >>>      ..ede/duply.net <http://duply.net> <http://duply.net>
>     >>>
>     >>>      On 06.03.2013 18:33, Cory Coager wrote:
>     >>>      > Bug ticket id is 1029516.
>     >>>      >
>     >>>      > On Wed, Mar 6, 2013 at 11:07 AM, Cory Coager <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>>> wrote:
>     >>>      >
>     >>>      >     The resultant files are indeed gzipped as I was able to 
> manually uncompress them. This may be related to the bug you speak of and I 
> updated the ticket yesterday.
>     >>>      >
>     >>>      >
>     >>>      >     On Wednesday, March 6, 2013,  <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>>> wrote:
>     >>>      >     > i see.. so it can be mistaken, cause it only tries to 
> interpret what file type it may be?
>     >>>      >     >
>     >>>      >     > anyway, sigtar, difftar should be untarable with tar, 
> right? so he could try that.
>     >>>      >     >
>     >>>      >     > or, just to find out if it compresses: backup a quite 
> big plain text/sql dump file.. the resulting volume will be significantly 
> smaller if it really compresses although forbidden.
>     >>>      >     >
>     >>>      >     > ..ede/duply.net <http://duply.net> <http://duply.net> 
> <http://duply.net>
>     >>>      >     >
>     >>>      >     > On 06.03.2013 16:27, Michael Terry wrote:
>     >>>      >     >> He means running "file XXX".
>     >>>      >     >>
>     >>>      >     >> "file" is a standard GNU/unix command to determine what 
> a file actually is (rather than just by filename).
>     >>>      >     >>
>     >>>      >     >> -mt
>     >>>      >     >>
>     >>>      >     >>
>     >>>      >     >> On 6 March 2013 04:18, <address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>> 
> <mailto:address@hidden <mailto:address@hidden> <mailto:address@hidden 
> <mailto:address@hidden>>> <mailto:address@hidden <mailto:address@hidden> 
> <mailto:address@hidden <mailto:address@hidden>> <mailto:address@hidden 
> <mailto:address@hidden> <mailto:address@hidden <mailto:address@hidden>>>>> 
> wrote:
>     >>>      >     >>
>     >>>      >     >>     On 06.03.2013 03:02, Cory Coager wrote:
>     >>>      >     >>     > I'm using --no-encryption and --no-compression 
> for a backup.  I noticed the files have a .difftar extension instead of 
> .difftar.gz. However, running a file against these shows that they are still 
> gzip'd with max compression.  Why is this happening?  Am I missing something?
>     >>>      >     >>     >
>     >>>      >     >>     > I'm using version 0.6.21 from Ubuntu ppa.
>     >>>      >     >>
>     >>>      >     >>
>     >>>      >     >>     shouldn't be.. how do you test? what do you mean by 
> " running a file against these "?
>     >>>      >     >>
>     >>>      >     >>     i wouldn't advise to use --no-compression. afaik it 
> has a bug in restoring currently.. check the launchpadpad bug tracker.
>     >>>      >     >>
>     >>>      >     >>     ..ede/duply.net <http://duply.net> 
> <http://duply.net> <http://duply.net> <http://duply.net>
>     >>>      >     >>


>     >
> 
>