Re: [Duplicity-talk] Fwd: GPG taking many ressources

[Henrik Olsen]

As I'm still having the issue, I was wondering if GPG 1.4.10 could be the 
issue, and I should try 2.x instead?

On Aug 9, 2010, at 10:22 PM, address@hidden wrote:

> no you can't. Thtat's the error's cause. Remove the first one or use a custom 
> pythonpath.
> 
> ede/duply.net
> 
> 
> On 09.08.2010 22:16, Henrik Olsen wrote:
>> I have the following in my python library path (I use export 
>> PYTHONPATH="[prefix]/lib/python":$PYTHONPATH):
>> 
>> l GnuPGInterface.*
>> -rw-r--r--  1 holsen  wheel  22000 Jan 11  2002 GnuPGInterface.py
>> -rw-r--r--  1 holsen  wheel  22022 Aug  9 22:00 GnuPGInterface.pyc
>> Henrik-Olsens-MacBook:python holsen$ l duplicity/GnuPGInterface.*
>> -rw-r--r--  1 holsen  wheel  22914 Jul 25 18:14 duplicity/GnuPGInterface.py
>> -rw-r--r--  1 holsen  wheel  23217 Aug  9 22:01 duplicity/GnuPGInterface.pyc
>> 
>> The last one in lib/python/duplicity is the one including 
>> 'threaded_waitpid'. 
>> 
>> Does this look correct? Or can I risk duplicity is using the on in 
>> lib/python and not lib/python/duplicity with the PYTHONPATH I use?
>> 
>> /Henrik
>> 
>> On Aug 9, 2010, at 8:17 PM, Kenneth Loafman wrote:
>> 
>>> Henrik,
>>> 
>>> Not sure how this is packaged on the Mac side, but you need to check to 
>>> make sure that the GnuPGInterface.py module is in the duplicity source.  I 
>>> had to make some mods to harvest the child processes and the changes were 
>>> not propagated.  You can tell if it is the correct module by looking for 
>>> the function 'threaded_waitpid' starting on line 660 of the module.
>>> 
>>> If the module does not coexist with duplicity, or the module is the wrong 
>>> one, just copy the correct one to the duplicity library and your problem 
>>> should go away.  The correct one is in the duplicity tarball.
>>> 
>>> ...Thanks,
>>> ...Ken
>>> 
>>> Henrik Olsen wrote:
>>>> Bump... No one with similar issues or comments? It's starting to become so 
>>>> much an issue on my installation, that I'm slowly being "forced" to drop 
>>>> my otherwise excellent duplicity solution. The CPU usage is getting out of 
>>>> hand, unfortunately.
>>>> 
>>>> Best regards
>>>> Henrik
>>>> 
>>>> Begin forwarded message:
>>>> 
>>>> 
>>>> 
>>>>> From: Henrik Olsen <address@hidden>
>>>>> 
>>>>> Date: August 3, 2010 8:19:11 PM GMT+02:00
>>>>> To: Discussion of the backup program duplicity 
>>>>> <address@hidden>
>>>>> 
>>>>> Subject: GPG taking many ressources
>>>>> 
>>>>> Using Duplicity 0.6.0.9 on Mac OS X 10.6.4 on a 2010 Macbook Pro.
>>>>> 
>>>>> I'm noticing that GPG is using a lot of CPU ressources. Seems to take 99% 
>>>>> of CPU for loooong period of times when running something like this
>>>>> 
>>>>> /usr/bin/python /usr/local/duplicity/bin/duplicity --name=[name] 
>>>>> --archive-dir=/var/cache/duplicity --full-if-older-than 3M 
>>>>> --asynchronous-upload --verbosity 4 --num-retries 5 --volsize=50 
>>>>> --include=/Users/[username] --exclude=/** / 
>>>>> ftp://
>>>>> [cut]
>>>>> 
>>>>> I see many many GPGs running.
>>>>> 
>>>>> ps aux | grep gpg
>>>>> root      2995  92.6  0.0  2448072   1160   ??  RN   Mon10AM 875:02.33 
>>>>> gpg --status-fd 112 --passphrase-fd 116 --logger-fd 109 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      3041   0.0  0.0  2436168    868   ??  SN   Mon10AM   0:05.08 
>>>>> gpg --logger-fd 118 --passphrase-fd 122 --batch --no-tty 
>>>>> --no-secmem-warning --force-mdc --symmetric
>>>>> root      2993   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.73 
>>>>> gpg --status-fd 108 --passphrase-fd 112 --logger-fd 105 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2990   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:04.08 
>>>>> gpg --status-fd 104 --passphrase-fd 108 --logger-fd 101 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2970   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:01.92 
>>>>> gpg --status-fd 100 --passphrase-fd 104 --logger-fd 97 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2967   0.0  0.0  2447048   1072   ??  SN   Mon10AM   0:00.46 
>>>>> gpg --status-fd 96 --passphrase-fd 100 --logger-fd 93 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2964   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:02.84 
>>>>> gpg --status-fd 92 --passphrase-fd 96 --logger-fd 89 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2941   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:00.39 
>>>>> gpg --status-fd 88 --passphrase-fd 92 --logger-fd 85 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2939   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:00.44 
>>>>> gpg --status-fd 84 --passphrase-fd 88 --logger-fd 81 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2937   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.52 
>>>>> gpg --status-fd 80 --passphrase-fd 84 --logger-fd 77 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2935   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:00.57 
>>>>> gpg --status-fd 76 --passphrase-fd 80 --logger-fd 73 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2927   0.0  0.0  2447048   1072   ??  SN   Mon10AM   0:00.43 
>>>>> gpg --status-fd 72 --passphrase-fd 76 --logger-fd 69 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2923   0.0  0.0  2447048   1084   ??  SN   Mon10AM   0:00.44 
>>>>> gpg --status-fd 68 --passphrase-fd 72 --logger-fd 65 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2920   0.0  0.0  2448072   1076   ??  SN   Mon10AM   0:00.55 
>>>>> gpg --status-fd 64 --passphrase-fd 68 --logger-fd 61 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2917   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.72 
>>>>> gpg --status-fd 60 --passphrase-fd 64 --logger-fd 57 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2893   0.0  0.0  2447572   1092   ??  SN   Mon10AM   0:01.20 
>>>>> gpg --status-fd 56 --passphrase-fd 60 --logger-fd 53 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2884   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.68 
>>>>> gpg --status-fd 52 --passphrase-fd 56 --logger-fd 49 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2879   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:01.01 
>>>>> gpg --status-fd 48 --passphrase-fd 52 --logger-fd 45 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2869   0.0  0.0  2448072   1076   ??  SN   Mon10AM   0:01.61 
>>>>> gpg --status-fd 44 --passphrase-fd 48 --logger-fd 41 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2857   0.0  0.0  2448072   1076   ??  SN   Mon10AM   0:01.95 
>>>>> gpg --status-fd 40 --passphrase-fd 44 --logger-fd 37 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2849   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.56 
>>>>> gpg --status-fd 36 --passphrase-fd 40 --logger-fd 33 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2830   0.0  0.0  2437832   1080   ??  SN   Mon10AM   0:01.86 
>>>>> gpg --status-fd 32 --passphrase-fd 36 --logger-fd 29 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2822   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:01.47 
>>>>> gpg --status-fd 28 --passphrase-fd 32 --logger-fd 25 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2815   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:00.45 
>>>>> gpg --status-fd 24 --passphrase-fd 28 --logger-fd 21 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2813   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:00.58 
>>>>> gpg --status-fd 20 --passphrase-fd 24 --logger-fd 17 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2803   0.0  0.0  2448072   1084   ??  SN   Mon10AM   0:00.48 
>>>>> gpg --status-fd 16 --passphrase-fd 20 --logger-fd 13 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2801   0.0  0.0  2448072   1084   ??  SN   Mon10AM   0:00.68 
>>>>> gpg --status-fd 12 --passphrase-fd 16 --logger-fd 9 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> root      2797   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:11.20 
>>>>> gpg --status-fd 7 --passphrase-fd 12 --logger-fd 5 --batch --no-tty 
>>>>> --no-secmem-warning --decrypt
>>>>> 
>>>>> Is this amount and ressource usage to be expected? Can anything be done 
>>>>> to lower it? It's getting quite annoying. I'm launching my backup scripts 
>>>>> via Launch Daemons. I've set nice and low priority i/o on the launch 
>>>>> daemon plist.
>>>>> 
>>>>> /Henrik
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> 
>>>> address@hidden
>>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>> 
>>>> 
>>>> 
>>>> 
>>> 
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>> 
>> 
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk