|
| From: | Roy Badami |
| Subject: | [Duplicity-talk] Problems with restore of LARGE backup from S3 |
| Date: | Mon, 23 Apr 2012 15:26:02 +0100 |
| User-agent: | Mozilla/5.0 (Windows NT 6.1; rv:11.0) Gecko/20120327 Thunderbird/11.0.1 |
I've made two attempts to perform a full test restore to another machine, and in both cases it eventually gives up. It got to volume 127 and then keeps failing with Download s3+http://...//duplicity-full.20120304T000513Z.vol127.difftar.gpg failed (attempt #9993, reason: IncompleteRead: IncompleteRead(0 bytes read, 1622573204 more expected))
Then eventually, when it hit the retry limit of 9999, it printed the following, and now is hung doing nothing
BackendException: Error downloading s3+http://...//duplicity-full.20120304T000513Z.vol127.difftar.gpg
A couple of other things I notice. The duplicity process has grown to almost 4GB - is this expected? (however, the machine has lots of RAM and swap, so this in itself shouldn't be a problem)
root 31336 9.1 1.8 3801340 613756 pts/2 Sl+ Apr18 673:44 /usr/bin/python /usr/bin/duplicity --s3-use-new-style --num-retries 9999 --tempdir /opt/restore-tmp --archive-dir /opt/restore-archive restore s3+http://.../ /opt/restore-test
Also, there are lots of gpg processes running (presumably left behind)? However no where near one per volume (there are 38 gpg processes, and we've processed 126 volumes before we hit the problem)
Any thoughts? Anyone else successfully using duplicity on filesystems this large?
root 461 0.0 0.0 123664 1704 pts/2 SL+ Apr18 0:03 gpg --status-fd 14 --passphrase-fd 18 --logger-fd 11 --batch --no-tty --no-secmem-warning --decrypt root 898 0.0 0.0 123664 1704 pts/2 SL+ Apr18 0:00 gpg --status-fd 18 --passphrase-fd 22 --logger-fd 15 --batch --no-tty --no-secmem-warning --decrypt root 1374 0.0 0.0 123664 1704 pts/2 SL+ Apr18 0:00 gpg --status-fd 22 --passphrase-fd 26 --logger-fd 19 --batch --no-tty --no-secmem-warning --decrypt root 1924 0.0 0.0 123664 1704 pts/2 SL+ Apr18 0:02 gpg --status-fd 26 --passphrase-fd 30 --logger-fd 23 --batch --no-tty --no-secmem-warning --decrypt root 2652 0.0 0.0 123664 1708 pts/2 SL+ Apr18 0:00 gpg --status-fd 30 --passphrase-fd 34 --logger-fd 27 --batch --no-tty --no-secmem-warning --decrypt root 3290 0.0 0.0 123664 1704 pts/2 SL+ Apr18 0:00 gpg --status-fd 34 --passphrase-fd 38 --logger-fd 31 --batch --no-tty --no-secmem-warning --decrypt root 3908 0.0 0.0 123664 1712 pts/2 SL+ Apr18 0:49 gpg --status-fd 38 --passphrase-fd 42 --logger-fd 35 --batch --no-tty --no-secmem-warning --decrypt root 4436 0.0 0.0 123664 1712 pts/2 SL+ Apr18 0:01 gpg --status-fd 42 --passphrase-fd 46 --logger-fd 39 --batch --no-tty --no-secmem-warning --decrypt root 4995 0.0 0.0 123664 1716 pts/2 SL+ Apr18 0:00 gpg --status-fd 46 --passphrase-fd 50 --logger-fd 43 --batch --no-tty --no-secmem-warning --decrypt root 5476 0.0 0.0 123664 1744 pts/2 SL+ Apr18 0:01 gpg --status-fd 50 --passphrase-fd 54 --logger-fd 47 --batch --no-tty --no-secmem-warning --decrypt root 6182 0.0 0.0 123664 1748 pts/2 SL+ Apr18 0:00 gpg --status-fd 54 --passphrase-fd 58 --logger-fd 51 --batch --no-tty --no-secmem-warning --decrypt root 6949 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:01 gpg --status-fd 58 --passphrase-fd 62 --logger-fd 55 --batch --no-tty --no-secmem-warning --decrypt root 7517 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:00 gpg --status-fd 62 --passphrase-fd 66 --logger-fd 59 --batch --no-tty --no-secmem-warning --decrypt root 8138 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:25 gpg --status-fd 66 --passphrase-fd 70 --logger-fd 63 --batch --no-tty --no-secmem-warning --decrypt root 8865 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:25 gpg --status-fd 70 --passphrase-fd 74 --logger-fd 67 --batch --no-tty --no-secmem-warning --decrypt root 9439 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:00 gpg --status-fd 74 --passphrase-fd 78 --logger-fd 71 --batch --no-tty --no-secmem-warning --decrypt root 10110 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:01 gpg --status-fd 78 --passphrase-fd 82 --logger-fd 75 --batch --no-tty --no-secmem-warning --decrypt root 10688 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:00 gpg --status-fd 82 --passphrase-fd 86 --logger-fd 79 --batch --no-tty --no-secmem-warning --decrypt root 11754 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:00 gpg --status-fd 86 --passphrase-fd 90 --logger-fd 83 --batch --no-tty --no-secmem-warning --decrypt root 12211 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:00 gpg --status-fd 90 --passphrase-fd 94 --logger-fd 87 --batch --no-tty --no-secmem-warning --decrypt root 12717 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:34 gpg --status-fd 94 --passphrase-fd 98 --logger-fd 91 --batch --no-tty --no-secmem-warning --decrypt root 13128 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:01 gpg --status-fd 98 --passphrase-fd 102 --logger-fd 95 --batch --no-tty --no-secmem-warning --decrypt root 13641 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:06 gpg --status-fd 102 --passphrase-fd 106 --logger-fd 99 --batch --no-tty --no-secmem-warning --decrypt root 14062 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:02 gpg --status-fd 106 --passphrase-fd 110 --logger-fd 103 --batch --no-tty --no-secmem-warning --decrypt root 14626 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:01 gpg --status-fd 110 --passphrase-fd 114 --logger-fd 107 --batch --no-tty --no-secmem-warning --decrypt root 15040 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:00 gpg --status-fd 114 --passphrase-fd 118 --logger-fd 111 --batch --no-tty --no-secmem-warning --decrypt root 15932 0.0 0.0 123664 1752 pts/2 SL+ Apr18 1:24 gpg --status-fd 122 --passphrase-fd 126 --logger-fd 119 --batch --no-tty --no-secmem-warning --decrypt root 16419 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:01 gpg --status-fd 126 --passphrase-fd 130 --logger-fd 123 --batch --no-tty --no-secmem-warning --decrypt root 16851 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:00 gpg --status-fd 130 --passphrase-fd 134 --logger-fd 127 --batch --no-tty --no-secmem-warning --decrypt root 17291 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:01 gpg --status-fd 134 --passphrase-fd 138 --logger-fd 131 --batch --no-tty --no-secmem-warning --decrypt root 17700 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:00 gpg --status-fd 138 --passphrase-fd 142 --logger-fd 135 --batch --no-tty --no-secmem-warning --decrypt root 18195 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:00 gpg --status-fd 142 --passphrase-fd 146 --logger-fd 139 --batch --no-tty --no-secmem-warning --decrypt root 18568 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:00 gpg --status-fd 146 --passphrase-fd 150 --logger-fd 143 --batch --no-tty --no-secmem-warning --decrypt root 19041 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:00 gpg --status-fd 150 --passphrase-fd 154 --logger-fd 147 --batch --no-tty --no-secmem-warning --decrypt root 19477 0.0 0.0 123664 1756 pts/2 SL+ Apr18 0:01 gpg --status-fd 154 --passphrase-fd 158 --logger-fd 151 --batch --no-tty --no-secmem-warning --decrypt root 19952 0.0 0.0 123664 1752 pts/2 SL+ Apr18 0:00 gpg --status-fd 158 --passphrase-fd 162 --logger-fd 155 --batch --no-tty --no-secmem-warning --decrypt root 25147 0.0 0.0 123664 1752 pts/2 SL+ Apr19 0:54 gpg --status-fd 9 --passphrase-fd 164 --logger-fd 4 --batch --no-tty --no-secmem-warning --decrypt root 28458 0.0 0.0 123664 1756 pts/2 SL+ Apr19 1:15 gpg --status-fd 118 --passphrase-fd 164 --logger-fd 5 --batch --no-tty --no-secmem-warning --decrypt
-- Roy Badami Roboreus Ltd Third Floor The Place 175 High Holborn London WC1V 7AA
| [Prev in Thread] | Current Thread | [Next in Thread] |