Re: A project-files implementation for Git projects

[Tassilo Horn]

Dmitry Gutov <address@hidden> writes:

Hi Dmitry,

>> where extra-includes works in addition to the standard VC ignore
>> rules (.gitignore, .hgignore).  Or do you want to override the
>> VC-internal rules?
>
> I'm afraid it might not work as well if we try to treat all (modified)
> ignores the same. In my understanding, the speed with which Git lists
> files to a large extent stems from not having to apply the ignores to
> the already-registered files.  Someone should benchmark this, but I
> think if we use the "negative pathspec" approach mentioned below for
> all ignores together, it might slow down file listing by an order of
> magnitude or several.
>
>> At least for Git and Hg, I came up with reasonable implementations:
>> [...]
>
> Terrific, thank you! How is Hg's performance with this approach? Does
> adding a few ignores (like 5 or 10) slow down the output measurably?

No, it doesn't slow down the listing (in comparison to just hg status
--all).  However, my test hg repo is not extraordinarily large (~4000
files).

> BTW, can Hg support extra whitelist entries as well?

"hg status --all" prints everything including ignored files.  An
--exclude restricts the output and filters the output so that matching
files are not listed.  --include also restricts the output so than only
files matched by such an include pattern are listed.

>> --8<---------------cut here---------------end--------------->8---
>> There's a semantic difference between Git and Hg in the treatment of
>> extra-ignores.  With Git, the extra-ignores do not rule out committed
>> files (i.e., they are only effective for untracked files) while for
>> Hg, they also rule out committed files.  I think the Hg semantics are
>> probably better
>
> Better and important, IMO.
>
>> but I don't see how to change the Git version so that it acts the
>> same way (except by re-filtering in lisp, of course), do you?
>
> Previously suggested:
>
> https://stackoverflow.com/questions/36753573/how-do-i-exclude-files-from-git-ls-files/53083343#53083343
>
> That means converting all extra-ignores into negative pathspec strings.

Ok, I see.  So that would be this and it seems like now we have the same
semantics as with the hg version:

--8<---------------cut here---------------start------------->8---
(defun vc-git-list-files (&optional dir
                                    include-unregistered
                                    extra-ignores)
  (let ((default-directory (or dir default-directory))
        (args '("-z")))
    (when include-unregistered
      (setq args (append args '("-c" "-o" "--exclude-standard"))))
    (when extra-ignores
      (setq args (append args
                         (cons "--"
                               (mapcar
                                (lambda (i)
                                  (format ":!:%s" i))
                                extra-ignores)))))
    (mapcar
     #'expand-file-name
     (cl-remove-if
      #'string-empty-p
      (split-string
       (apply #'vc-git--run-command-string nil "ls-files" args)
       "\0")))))
--8<---------------cut here---------------end--------------->8---

So basically "git status ... -- '*.el'" corresponds to "hg status --all
--include '*.el'" whereas negative pathspecs correspond to hg's
--exclude.

A quick look at bzr suggests there's just a way restrict positively,
i.e., like --include with hg.

>> I haven't looked at the other backends.  I guess bzr will probably be
>> doable, too.  However, for SVN, there's no way to list unregistered
>> files.  A correct (but horribly slow) default implementation should
>> also be doable.
>
> Yeah, I wonder if we should treat this as a VC operation. On the other
> hand, the fallback implementation could just as well use 'find'.

Right now, it uses `vc-file-tree-walk'...

Bye,
Tassilo