master b19259c: Futher tramp-crypt implementation and documentation

[Michael Albinus]

branch: master
commit b19259c8412ee2e715c4bd145711e23729411fd0
Author: Michael Albinus <michael.albinus@gmx.de>
Commit: Michael Albinus <michael.albinus@gmx.de>

    Futher tramp-crypt implementation and documentation
    
    * doc/misc/tramp.texi (Top, Configuration): Insert section
    `Keeping files encrypted' in menu.
    (Keeping files encrypted): New node.
    
    * lisp/net/tramp-crypt.el (tramp-crypt-file-name-handler-alist):
    Add `tramp-set-file-uid-gid'.
    (tramp-crypt-maybe-open-connection): Simplify.
    (tramp-crypt-do-encrypt-or-decrypt-file): Use `binary' coding system.
    (tramp-crypt-handle-set-file-uid-gid): New defun.
    
    * test/lisp/net/tramp-tests.el (tramp-test09-insert-file-contents):
    Adapt test.
---
 doc/misc/tramp.texi          | 118 +++++++++++++++++++++++++++++++++++++++++--
 lisp/net/tramp-crypt.el      |  19 ++++---
 test/lisp/net/tramp-tests.el |  25 ++++++---
 3 files changed, 144 insertions(+), 18 deletions(-)

diff --git a/doc/misc/tramp.texi b/doc/misc/tramp.texi
index d1688de..176d3a5 100644
--- a/doc/misc/tramp.texi
+++ b/doc/misc/tramp.texi
@@ -141,6 +141,7 @@ Configuring @value{tramp} for use
 * Remote shell setup::          Remote shell setup hints.
 * Android shell setup::         Android shell setup hints.
 * Auto-save and Backup::        Auto-save and Backup.
+* Keeping files encrypted::     Protect remote files by encryption.
 * Windows setup hints::         Issues with Cygwin ssh.
 
 Using @value{tramp}
@@ -667,6 +668,7 @@ might be used in your init file:
 * Remote shell setup::          Remote shell setup hints.
 * Android shell setup::         Android shell setup hints.
 * Auto-save and Backup::        Auto-save and Backup.
+* Keeping files encrypted::     Protect remote files by encryption.
 * Windows setup hints::         Issues with Cygwin ssh.
 @end menu
 
@@ -2648,6 +2650,114 @@ auto-saved files to the same directory as the original 
file.
 Alternatively, set the user option @code{tramp-auto-save-directory}
 to direct all auto saves to that location.
 
+
+@node Keeping files encrypted
+@section Protect remote files by encryption
+@cindex Encrypt remote directories
+
+Sometimes, it is desirable to protect files located on remote
+directories, like cloud storages.  In order to do this, you might
+instruct @value{tramp} to encrypt all files copied to a given remote
+directory, and to decrypt such files when accessing.  This includes
+both file contents and file names.
+
+@value{tramp} does this transparently.  Although both files and file
+names are encrypted on the remote side, they are accessible inside
+Emacs as they wouldn't be transformed as such.
+
+@cindex @command{encfs}
+@cindex @command{encfsctl}
+Internally, @value{tramp} uses the @command{encfs} package.
+Therefore, this feature is available only if this package is installed
+on the local host.  @value{tramp} does not keep and @samp{encfs
+mountpoint} permanently.  Instead, it encrypts / decrypts files and
+file names on the fly, using @command{encfsctl}.
+
+@deffn Command tramp-crypt-add-directory name
+This command marks the existing remote directory @var{name} for
+encryption.  Files in that directory and all subdirectories will be
+encrypted before copying to, and decrypted after copying from that
+directory.  File and directory names will be also encrypted.
+@end deffn
+
+@defopt tramp-crypt-encfs-option
+If a remote directory is marked for encryption, it is initialized via
+@command{encfs} the very first time a file in this directory is
+accessed.  This user option controls, which default @command{encfs}
+configuration option will be selected, it can be @t{"--standard"}
+or @t{"--paranoia"}.  See the @samp{encfs(1)} man page for details.
+
+However, @value{tramp} must adapt these configuration sets.  The
+@code{chainedNameIV} configuration option must be disabled; otherwise
+@value{tramp} couldn't handle file name encryption transparently.
+@end defopt
+
+A password protected @option{encfs} configuration file is created the
+very first time you access an encrypted remote directory.  It is kept
+in your @code{user-emacs-directory} with the url-encoded directory
+name as part of the basename, and @file{encfs6.xml} as suffix.  If
+you, for example, mark the remote directory
+@file{@trampfn{nextcloud,user@@host,/path/to/dir}} for encryption, the
+configuration file is saved as
+@file{tramp-%2Fnextcloud%3Auser%40host%3A%2Fpath%2Fto%2Fdir%2F.encfs6.xml}
+in @code{user-emacs-directory}.  Do not loose this file and the
+corresponding password; otherwise there is no way to decrypt your
+encrypted files.
+
+@defopt tramp-crypt-save-encfs-config-remote
+If this user option is non-nil (the default), the @option{encfs}
+configuration file @file{.encfs6.xml} is also kept in the encrypted
+remote directory.  It depends on you, whether you regard the password
+protection of this file as sufficient.  The advantage would be, that
+such a remote directory could be accessed by different Emacs sessions,
+different users, without presharing the configuration file between the
+users.
+@end defopt
+
+The command @command{encfsctl}, the workhorse for encryption /
+decryption, needs the configuration file password every call.
+Therefore, it is recommend to cache this password in Emacs.  This can
+be done using @code{auth-sources}, @ref{Using an authentication file}.
+An entry needs the url-encoded directory name as machine, your local
+user name as user, and the password.  The port is optional, if given
+it must be the string @t{"crypt"}.  The example above would require
+the following entry in the authentication file (@t{"yourname"} is the
+result of @code{(user-login-name)}):
+
+@example
+machine %2Fnextcloud%3Auser%40host%3A%2Fpath%2Fto%2Fdir%2F \
+        login yourname port crypt password geheim
+@end example
+
+If you use a remote file name with a quoted localname part, this
+localname and the corresponding file will not be encrypted /
+decrypted.  If you have an encrypted remote directory
+@file{@trampfn{nextcloud,user@@host,/path/to/dir}}, the command
+
+@example
+@kbd{C-x d @trampfn{nextcloud,user@@host,/path/to/dir}}
+@end example
+
+@noindent
+will show the directory listing with the plain file names, and the
+command
+
+@example
+@kbd{C-x d @trampfn{nextcloud,user@@host,/:/path/to/dir}}
+@end example
+
+@noindent
+will show the directory listing with the encrypted file names, and
+visiting a file will show its encrypted contents.  However, it is
+highly discouraged to mix encrypted and not encrypted files in the
+same directory.
+
+@deffn Command tramp-crypt-add-directory name
+If a remote directory shall not include encrypted files anymore, it
+must be indicated by this command.
+@end deffn
+
+
 @node Windows setup hints
 @section Issues with Cygwin ssh
 @cindex cygwin, issues
@@ -2681,10 +2791,10 @@ Wiki} it is explained how to use the helper program
 @cindex @option{scpx} method with cygwin
 
 When using the @option{scpx} access method, Emacs may call
-@command{scp} with MS Windows file naming, such as @code{c:/foo}.  But
+@command{scp} with MS Windows file naming, such as @file{c:/foo}.  But
 the version of @command{scp} that is installed with Cygwin does not
 know about MS Windows file naming, which causes it to incorrectly look
-for a host named @code{c}.
+for a host named @samp{c}.
 
 A workaround: write a wrapper script for @option{scp} to convert
 Windows file names to Cygwin file names.
@@ -4158,8 +4268,8 @@ Host *
 @end group
 @end example
 
-Check @command{man ssh_config} whether these options are supported on
-your proxy host.
+Check the @samp{ssh_config(5)} man page whether these options are
+supported on your proxy host.
 
 
 @item
diff --git a/lisp/net/tramp-crypt.el b/lisp/net/tramp-crypt.el
index d9ba2e4..664f441 100644
--- a/lisp/net/tramp-crypt.el
+++ b/lisp/net/tramp-crypt.el
@@ -44,11 +44,11 @@
 
 ;; If the user option `tramp-crypt-save-encfs-config-remote' is
 ;; non-nil (the default), the encfs configuration file ".encfs6.xml"
-;; is also be kept in the crypted remote directory.  It depends,
+;; is also kept in the crypted remote directory.  It depends on you,
 ;; whether you regard the password protection of this file as
 ;; sufficient.
 
-;; If you apply an operation with a quoted localname part, this
+;; If you use a remote file name with a quoted localname part, this
 ;; localname and the corresponding file will not be encrypted/
 ;; decrypted.  For example, if you have a crypted remote directory
 ;; "/nextcloud:user@host:/crypted_dir", the command
@@ -213,7 +213,7 @@ If NAME doesn't belong to a crypted remote directory, retun 
nil."
     (start-file-process . ignore)
     ;; `substitute-in-file-name' performed by default handler.
     ;; (temporary-file-directory . tramp-crypt-handle-temporary-file-directory)
-    ;; `tramp-set-file-uid-gid' performed by default handler.
+    (tramp-set-file-uid-gid . tramp-crypt-handle-set-file-uid-gid)
     ;; (unhandled-file-name-directory . ignore)
     (vc-registered . ignore)
     (verify-visited-file-modtime . tramp-handle-verify-visited-file-modtime)
@@ -334,7 +334,6 @@ connection if a previous connection has died for some 
reason."
          (with-temp-file local-config
            (insert-file-contents
             (expand-file-name tramp-crypt-encfs-config tmpdir1))
-           (goto-char (point-min))
            (when (search-forward
                   "<chainedNameIV>1</chainedNameIV>" nil 'noerror)
              (replace-match "<chainedNameIV>0</chainedNameIV>")))
@@ -427,9 +426,9 @@ If OP ist `decrypt', the basename of INFILE must be an 
encrypted file name."
             (dir (tramp-crypt-file-name-p root))
             (crypt-vec (tramp-crypt-dissect-file-name dir)))
     (let ((coding-system-for-read
-          (if (eq op 'decrypt) 'raw-text coding-system-for-read))
+          (if (eq op 'decrypt) 'binary coding-system-for-read))
          (coding-system-for-write
-          (if (eq op 'encrypt) 'raw-text coding-system-for-write)))
+          (if (eq op 'encrypt) 'binary coding-system-for-write)))
       (tramp-crypt-send-command
        crypt-vec "cat" (and (eq op 'encrypt) "--reverse")
        (file-name-directory infile)
@@ -759,6 +758,14 @@ absolute file names."
       (tramp-compat-set-file-times
        (tramp-crypt-encrypt-file-name filename) time flag))))
 
+(defun tramp-crypt-handle-set-file-uid-gid (filename &optional uid gid)
+  "Like `tramp-set-file-uid-gid' for Tramp files."
+  (with-parsed-tramp-file-name filename nil
+    (tramp-flush-file-properties v localname)
+    (let (tramp-crypt-enabled)
+      (tramp-set-file-uid-gid
+       (tramp-crypt-encrypt-file-name filename) uid gid))))
+
 (add-hook 'tramp-unload-hook
          (lambda ()
            (unload-feature 'tramp-crypt 'force)))
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index d578c35..578da41 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -2294,16 +2294,25 @@ This checks also `file-name-as-directory', 
`file-name-directory',
       (unwind-protect
          (with-temp-buffer
            (write-region "foo" nil tmp-name)
-           (insert-file-contents tmp-name)
-           (should (string-equal (buffer-string) "foo"))
-           (insert-file-contents tmp-name)
-           (should (string-equal (buffer-string) "foofoo"))
+           (let ((point (point)))
+             (insert-file-contents tmp-name)
+             (should (string-equal (buffer-string) "foo"))
+             (should (= point (point))))
+           (goto-char (1+ (point)))
+           (let ((point (point)))
+             (insert-file-contents tmp-name)
+             (should (string-equal (buffer-string) "ffoooo"))
+             (should (= point (point))))
            ;; Insert partly.
-           (insert-file-contents tmp-name nil 1 3)
-           (should (string-equal (buffer-string) "oofoofoo"))
+           (let ((point (point)))
+             (insert-file-contents tmp-name nil 1 3)
+             (should (string-equal (buffer-string) "foofoooo"))
+             (should (= point (point))))
            ;; Replace.
-           (insert-file-contents tmp-name nil nil nil 'replace)
-           (should (string-equal (buffer-string) "foo"))
+           (let ((point (point)))
+             (insert-file-contents tmp-name nil nil nil 'replace)
+             (should (string-equal (buffer-string) "foo"))
+             (should (= point (point))))
            ;; Error case.
            (delete-file tmp-name)
            (should-error