Re: project.el semantics

[Dmitry Gutov]

On 11/22/2015 11:27 PM, John Wiegley wrote:

>    - All files containing source code (i.e., no generated content)

And no txt/markdown/jpeg/object/etc files, right?

>    - All files under version control

That's unrelated to the project's configuration. And anyway, I don't 
think you can implement this in any way except by taking all files, and 
calling vc-responsible-backend on each of them, one by one.

>    - All generated files and directories
>    - The arguments one would provide to GNU find to produce a similar list of
>      files to any of the above
>    - The directories and files relating to the project, with some indication of
>      whether those directories should be recursed (and how)

Makes sense.

Also, the lists of files should really match the corresponding lists of 
directories in some way, wouldn't you say? Recursive traversal or not, 
as an API consumer I'd be surprised if "list of source code directories" 
tells me one thing, and "list of source code files" tells me something 
different (files from directories not mentioned in the former list, or 
missing files from the said directories, for non-obvious reasons).

So ideally, the way a project backend defines the information should be 
as concise and unambiguous as possible.

> To achieve this, we take the "project set" to be intensionally defined, and
> provide an API for querying along the multiple views. The data returned from
> these views should be generative (using stream.el), so that element-by-element
> actions do not require excess allocation up front.

It would help to understand the use cases where stream.el would help, first.

We should also evaluate the overhead that stream.el (or generator.el, 
which would be my preferred approach to this) will add to listing all 
files in a big project. No excess allocation up front is nice, but the 
operations we provide must be fast.

> For convenience, the DEFINITION can often be determined from context, such as
> by querying Git or VC.

We'll be reading project files for that. Or some third-party project 
backends will, at least.

> For convenience, the QUERY can be reduced to a set of possibilities, like
> "source code", or "ignored files", or "library files", etc.

A set of combined conditions, I'd say. "source code inside the project 
roots", "ignored files in directory D", "library files anywhere". 
(What's library files? We'll probably avoid having this category).

> While the surface API might encode these conveniences as short-hands for what
> are likely to be common uses, the underlying API should allow the general form
> of a rich query. The project is, in a sense, a database of entities, and it's
> hard to pre-determine all possible useful queries of such data.

We should define a set of basic, orthogonal axes of information a 
project can tell about it, and then add to it later. Preferably in areas 
orthogonal to the already defined ones.

> How these conveniences are implemented is completely open; `cl-defgeneric'
> methods that encode the QUERY&SELECTION against an auto-determined DEFINITION
> is just fine, and is how project.el is written today.

If project.el satisfies our basic QUERY&SELECTION needs, we could as 
well document it already, because the "more flexible ways" would just 
build on top of it, as a set of utility functions (right?).

On the other hand, I'm not sure we've reached that consensus.