Re: [Duplicity-talk] Duplicity + asymmetric encryption

[edgar . soldin]

On 08.09.2015 00:56, Marko Božiković wrote:
> On 07.09.2015. 12:08, address@hidden wrote:
>> On 07.09.2015 00:24, Marko Božiković wrote:
>>> On 03.08.2015. 3:31, address@hidden wrote:
>>>>
>>>>
>>>> On August 2, 2015 11:41:31 PM GMT+02:00, "Marko Božiković" 
>>>> <address@hidden> wrote:
>>>>> Hi all,
>>>>>
>>>>> I've started playing with Duplicity for my cloud backup needs and I
>>>>> have a few
>>>>> questions regarding asymmetric encryption with gpg. Apologies for a
>>>>> long mail,
>>>>> but I'm confused by Duplicity's behaviour.
>>>>>
>>>>> I'm using Debian Jessie, duply 1.9.2, duplicity 0.7.3 and gpg 2.1.6
>>>>> (these are
>>>>> installed from testing/experimental repos)
>>>>>
>>>>> I've created a symbolic link gpg->gpg2 in order to force duplicity to
>>>>> use gpg2.
>>>>>
>>>>> I have a gpg2 keyring setup with a master 4096 bit key, three 2048 bit
>>>>> subkeys
>>>>> (for signing, encryption and authentication) for everyday use that are
>>>>> stored
>>>>> on a smart card and a newly created 2048 bit (S, E) subkey for
>>>>> duplicity backup.
>>>>>
>>>>> I keep my secret master key offline. The keyring on my laptop only
>>>>> contains
>>>>> the master public key and stubs for secret subkeys that live on the
>>>>> smartcard.
>>>>> The keyring on my backup machine only contains the public master key
>>>>> and the
>>>>> duplicity backup subkey.
>>>>>
>>>>> Here are the relevant bits from my duply config (I've replaced my
>>>>> backup key's
>>>>> fingerprint with YYYYYYYY:
>>>>>
>>>>> GPG_KEY='YYYYYYYY'
>>>>> GPG_OPTS='--pinentry-mode loopback'
>>>>> VOLSIZE=250
>>>>> DUPL_PARAMS="$DUPL_PARAMS --volsize $VOLSIZE "
>>>>> DUPL_PARAMS="$DUPL_PARAMS --file-prefix-archive=backup-data-
>>>>> --file-prefix-manifest=includes- --file-prefix-signature=signatures-"
>>>>>
>>>>> Gpg 2.1 always uses gpg-agent for secret key access and it doesn't set
>>>>> GPG_AGENT_INFO env. variable. This is the output I get when I run duply
>>>>> with
>>>>> no GPG_AGENT_INFO set (YYYYYYYY is my backup key's fingerprint and
>>>>> XXXXXXXX is
>>>>> my master key fingerprint)
>>>>>
>>>>> -------------------------------
>>>>> -- BEGIN no GPG_AGENT_INFO set
>>>>> -------------------------------
>>>>> Start duply v1.9.2, time is 2015-08-02 22:02:48.
>>>>> Using profile '/home/bozho/.duply/photo'.
>>>>> Using installed duplicity version 0.7.03, python 2.7.10, gpg 2.1.6
>>>>> (Home:
>>>>> ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan',
>>>>> grep
>>>>> '(GNU grep) 2.20', bash '4.3.30(1)-release (x86_64-pc-linux-gnu)'.
>>>>> Autoset found secret key of first GPG_KEY entry 'YYYYYYYY' for signing.
>>>>> Cannot use gpg-agent. GPG_AGENT_INFO not set.
>>>>> Checking TEMP_DIR '/tmp' is a folder (OK)
>>>>> Checking TEMP_DIR '/tmp' is writable (OK)
>>>>> Enter passphrase: <MY SECRET KEY PASSWORD>
>>>>> Test - Encrypt to 'YYYYYYYY' & Sign with 'YYYYYYYY' (OK)
>>>>> Test - Decrypt (OK)
>>>>> Test - Compare (OK)
>>>>> Cleanup - Delete '/tmp/duply.1629.1438549369_*'(OK)
>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>
>>>>> WARNING:
>>>>>
>>>>> Backup failed.
>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>> passphrase
>>>>> given - skipped
>>>>>
>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>
>>>>> WARNING:
>>>>>
>>>>> Backup failed.
>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>> passphrase
>>>>> given - skipped
>>>>>
>>>>>
>>>>> --- Start running command PRE at 22:03:14.018 ---
>>>>> Skipping n/a script '/home/bozho/.duply/photo/pre'.
>>>>> --- Finished state OK at 22:03:14.122 - Runtime 00:00:00.103 ---
>>>>>
>>>>> --- Start running command BKP at 22:03:14.135 ---
>>>>> Reading globbing filelist /home/bozho/.duply/photo/exclude
>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>> Last full backup date: none
>>>>> GnuPG passphrase: <ANY PASSPHRASE>
>>>>> GnuPG passphrase for signing key: <MY SECRET KEY PASSWORD>
>>>>> No signatures found, switching to full backup.
>>>>> -------------------------------
>>>>> -- END no GPG_AGENT_INFO set
>>>>> -------------------------------
>>>>>
>>>>>
>>>>> Note that I get asked a passphrase three times:
>>>>>
>>>>> - "Enter passphrase", which requires my secret backup subkey password -
>>>>> why is
>>>>> that needed here?
>>>>>
>>>>> - "GnuPG passphrase:", which can be any password - I'm guessing that's
>>>>> used as
>>>>> the session key that actually encrypts the data and then gets encrypted
>>>>> by my
>>>>> public subkey. Why is this necessary? In OpenPGP, the session key
>>>>> can/should
>>>>> be a long random key - the recipient doesn't need to know it.
>>>>>
>>>>> - "GnuPG passphrase for signing key" - ok, this one is for signing,
>>>>> since it's
>>>>> enabled and I'm using the same key.
>>>>>
>>>>>
>>>>>
>>>>> Now, if I restart gpg-agent in order to reset its caching and actually
>>>>> export
>>>>> GPG_AGENT_INFO, I get a slightly different output:
>>>>>
>>>>> -------------------------------
>>>>> -- BEGIN GPG_AGENT_INFO set
>>>>> -------------------------------
>>>>> Start duply v1.9.2, time is 2015-08-02 22:12:42.
>>>>> Using profile '/home/bozho/.duply/photo'.
>>>>> Using installed duplicity version 0.7.03, python 2.7.10, gpg 2.1.6
>>>>> (Home:
>>>>> ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright(C) Michael D. Brennan',
>>>>> grep
>>>>> '(GNU grep) 2.20', bash '4.3.30(1)-release (x86_64-pc-linux-gnu)'.
>>>>> Autoset found secret key of first GPG_KEY entry 'YYYYYYYY' for signing.
>>>>> Autoenable use of gpg-agent. GPG_PW or GPG_PW_SIGN (enc != sign key)
>>>>> not set.
>>>>> Checking TEMP_DIR '/tmp' is a folder (OK)
>>>>> Checking TEMP_DIR '/tmp' is writable (OK)
>>>>> Enter passphrase: <MY SECRET KEY PASSWORD>
>>>>> Test - Encrypt to 'YYYYYYYY' & Sign with 'YYYYYYYY' (OK)
>>>>> Test - Decrypt (OK)
>>>>> Test - Compare (OK)
>>>>> Cleanup - Delete '/tmp/duply.4740.1438549962_*'(OK)
>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>
>>>>> WARNING:
>>>>>
>>>>> Backup failed.
>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>> passphrase
>>>>> given - skipped
>>>>>
>>>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>>>
>>>>> WARNING:
>>>>>
>>>>> Backup failed.
>>>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No
>>>>> passphrase
>>>>> given - skipped
>>>>>
>>>>>
>>>>> --- Start running command PRE at 22:13:22.870 ---
>>>>> Skipping n/a script '/home/bozho/.duply/photo/pre'.
>>>>> --- Finished state OK at 22:13:22.956 - Runtime 00:00:00.085 ---
>>>>>
>>>>> --- Start running command BKP at 22:13:22.966 ---
>>>>> Reading globbing filelist /home/bozho/.duply/photo/exclude
>>>>> Local and Remote metadata are synchronized, no sync needed.
>>>>> Last full backup date: none
>>>>> No signatures found, switching to full backup.
>>>>> -------------------------------
>>>>> -- END GPG_AGENT_INFO set
>>>>> -------------------------------
>>>>>
>>>>> Notice that I get asked for a password only once, at the beginning
>>>>> ("Enter
>>>>> passphrase") Why is that, and is the session key being randomly
>>>>> generated in
>>>>> this case or is my secret subkey password used as a session key (this
>>>>> is a bad
>>>>> idea)
>>>>>
>>>>> Thank you,
>>>>
>>>>
>>>> probably 
>>>>  https://sourceforge.net/p/ftplicity/feature-requests/36/
>>>>
>>>> try setting --use-agent manually in conf.
>>>>
>>>> ..ede/duply.net
>>>>
>>>
>>>
>>> Hi,
>>>
>>> Sorry for a late reply.
>>>
>>> I tried using your suggestion and added --use-agent to DUPL_PARAMS, but it
>>> didn't help.
>>>
>>> I've tried with and without setting GPG_AGENT_INFO variable, I get the same
>>> output after being asked for the key password:
>>>
>>>
>>>
>>> Start duply v1.9.2, time is 2015-09-05 22:45:50.
>>> Using profile '/home/bozho/.duply/photo'.
>>> Using installed duplicity version 0.7.03, python 2.7.10, gpg 2.1.6 (Home:
>>> ~/.gnupg), awk 'mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan', grep
>>> '(GNU grep) 2.21', bash '4.3.39(1)-release (x86_64-pc-linux-gnu)'.
>>> Autoset found secret key of first GPG_KEY entry 'YYYYYYYY' for signing.
>>> Autoenable use of gpg-agent. GPG_PW or GPG_PW_SIGN (enc != sign key) not 
>>> set.
>>> Checking TEMP_DIR '/tmp' is a folder (OK)
>>> Checking TEMP_DIR '/tmp' is writable (OK)
>>> Test - Encrypt to 'YYYYYYYY' & Sign with 'YYYYYYYY' (OK)
>>> Test - Decrypt (OK)
>>> Test - Compare (OK)
>>> Cleanup - Delete '/tmp/duply.3034.1441489550_*'(OK)
>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>
>>> WARNING:
>>>
>>> Backup failed.
>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No passphrase
>>> given - skipped
>>>
>>> Export SEC key 'YYYYYYYY' (FAILED)
>>>
>>> WARNING:
>>>
>>> Backup failed.
>>> gpg: key XXXXXXXX/YYYYYYYY: error receiving key from agent: No passphrase
>>> given - skipped
>>>
>>>
>>> --- Start running command PRE at 22:45:54.331 ---
>>> Skipping n/a script '/home/bozho/.duply/photo/pre'.
>>> --- Finished state OK at 22:45:54.410 - Runtime 00:00:00.079 ---
>>>
>>> --- Start running command BKP at 22:45:54.419 ---
>>> Reading globbing filelist /home/bozho/.duply/photo/exclude
>>> Local and Remote metadata are synchronized, no sync needed.
>>> Last full backup date: none
>>> No signatures found, switching to full backup.
>>>
>>>
>>>
>>>
>>> However, the backup seems to proceed normally...
>>>
>>> Thank you,
>>>
>>
>> can i see your duply conf file? obfuscate private strings in it beforehand.
> 
> Here you go.
> 
> Thank you for persevering with this :)
> 
> Just as a note (I don't remember if I've mentioned this before), I moved my
> /usr/bin/gpg to /usr/bin/gpg1 and created /usr/bin/gpg symlink to gpg2.
> 
> 

i notice you do not set the gpg passphrase. this is the classic way to provide 
it, the other being gpg-agent. 

for testing purposes please try and set GPG_PW and see if it works as expected. 
don't be confused that you need the gpg-agent parameters 
(allow-loopback-pinentry in your .gnupg/gpg-agent.conf and 
GPG_OPTS='--pinentry-mode loopback') for that to work as gpg-agent is now 
(since gpg2.1) the "unified" passphrase input interface to gpg2.1.

alternatively you can try the gpg-agent route. therefor you remove the GPG_OPTS 
conf param and add DUPL_PARAMS="$DUPL_PARAMS --use-agent". i didn't come around 
to implement a senseful route to deal with the gpg2.1 changes so for for duply.

another thought. not sure that the gpg symlinking does something weird here. 
please use duplicity 0.7.04 with the new --gpg-binary parameter.

as per why and when the passphrases/keys are needed let's move these questions 
to after we solved your current issue.

btw. you could always use gpg2.0-- and it should work right out of the box. why 
do you insist on gpg2.1?

..ede/duply.net