More efficient tramp-handle-file-name-all-completions

[Julian Scheid]

The attached patch improves several aspects of
tramp-handle-file-name-all-completions.

It combines two remote operations, cd and ls, into a single operation.

It sets the file-exists-p cache for the directory and for every file
in the directory (in case of success). This gets rid of a bunch of
file-exists-p remote ops that completion-pcm-try-completion performs
after file-name-all-completions.

In addition, with this patch tramp-handle-file-name-all-completions
will only grab relevant entries from the remote directory. For
example, /usr/bin/zip<TAB> will only "ls zip*", reducing, in this
example, the amount of data to be transferred from 25K for all of my
remote /usr/bin to only a few bytes.

The assumption made here is that the remote ls supports the -d flag. I
am not aware of any ls that don't but then again, I don't know all
*ices. If this was a problem, tramp could detect whether the remote ls
does support -d and not use wildcards where it doesn't.

Partial directory fetches requires a slightly more complex caching
mechanism that is best described by an example: "/usr/bin/zip<TAB>"
will put the the possible completions for "zip" into the cache using
key "/usr/bin/zip". Then, /usr/bin/zipc<TAB> will look for cache
entries for keys /usr/bin/zipc, /usr/bin/zip, /usr/bin/zi, /usr/bin/z
and /usr/bin/, using the first match it can find. In this case
/usr/bin/zip will match, the cache entries of which are a superset of
the entries we were looking for.

Another change in this patch is that the error message for when the
directory can't be cd'ed to is taken from cd's stderr. In my test
environment the error will now read something like
"tramp-handle-file-name-all-completions: /foo: No such file or
directory." This is a bit more informative and more consistent with
what file-name-all-completions does.

I've added copious amounts of commentary that is meant for anyone
reviewing the patch, not necessarily meant to be carried into the
repository.

One further improvement I can think of is to use find(1) or perl on
the remote host in place of ls to avoid using shell expansion for
wildcards which can be inefficient when there is a large number of
matches.

Also, when attempting to complete on a non-existent prefix there are a
few file-exists-p that are currently not answered from the cache. This
could be improved by making file-exists-p look into
file-name-all-completions cache entries (e.g. existence of
/usr/bin/zipfoo can be answered when there's a non-expired cache entry
for /usr/bin/zip*.)

tramp-handle-file-name-all-completions-optimization.patch
Description: Binary data