Re: A project-files implementation for Git projects

[Dmitry Gutov]

Hi Tassilo,

On 16.09.2019 16:32, Tassilo Horn wrote:

> Ah, "hg status --all" lists all files including their state (untracked,
> ignored, you-name-it), so that's the one we should use.  Performance
> seems to be the same as for "hg files".

In my testing the performance difference is about 2x:

$ bash -c "time hg status -c >/dev/null"

real    0m12,015s
user    0m1,899s
sys     0m10,113s

$ bash -c "time hg files >/dev/null"

real    0m5,970s
user    0m1,004s
sys     0m4,965s

(project-files (project-current)) takes ~7 seconds here on the same repo 
(Mozilla Firefox checkout).

But if it's faster than 'find' anyway on some platforms, why not? As 
long as there's a solution that will handle the adjusted ignore rules in 
a similarly performant fashion.

> I think we can come up with a VC list-files operation which optionally
> includes untracked and ignored files (where the latter implies the
> former, doesn't it?)

Whether it implies or not, depends on which set of ignores we're talking 
about (Git's own or the modified one).

> but I'd leave the filtering according to
> project-vc-ignores to project.el.

Have you tries benchmarking this approach? E.g. calling 'git ls-files -c 
-o -z' and then doing all the filtering indicated by .gitignore rules?

Try it on the current Emacs repo.

IME it's the ignore rules that take up 99% of the CPU time when using 
'find'. Without them, 'find .' is instant (though that depends on the 
disk access speed). If we're going to implement that in Elisp, I'd wager 
it's going to be even slower.

> How would project.el call such a VC list-files operation?  I guess you
> would include untracked and also ignored files, right?

I encourage you to try this approach even with Git only, without the VC 
facade, and see where we end up.

> In my use-case,
> the inclusion of ignored files would probably increase the size of the
> list of files by a factor of at least 2 because for every *.java file in
> our project, there's at least one ignored *.class file (but probably
> more like 2-20 *.class files).

I'm fairly sure the compilation artefacts aren't going to be the only 
problem of this approach.

> So a new defcustom or include ignored
> files only if project-vc-ignores has a non-nil value?

I suppose we could end up having different branches of logic for whether 
the user ends up using this variable, but I'd rather not if the "yes" 
branch is always going to be slower. It's like punishing them for using 
a reasonable feature.