Re: new-flex-completion-style

[João Távora]

Stefan Monnier <address@hidden> writes:

> W.r.t the UI sorting minibuffer.el has 2 different UIs with 2 different
> sorting behaviors: there's the *Completions* buffer where results are
> sorted alphabetically, and there's the force-complete cycling which
> sorts by most-recently seen (using the minibuffer history variable) and
> by length.

Tangent: When using icomplete (who else here uses icomplete?), this is
(the length) not good at all for switching buffers.  I want it to behave
like ido, where the natural order of buffer-list is preserved when the
buffer can't be found in the minibuffer-history-variable.  IOW for the
icomplete UI, sorting by length makes little sense.  It may make sense
for completion-at-point cycling.  I'm working on a patch for this.

> All this was designed in the context of completion tables and styles
> where all returned candidates are basically equally likely.
>
> Flex is going in the direction of completion systems where instead of
> splitting the candidates between "possible matches" and "impossible
> matches", the matches are all considered as possible, with some more
> likely than others.
> So in such a context, it's of utmost importance for the completion
> table-or-style to provide its own sorting.  And in that case, I guess
> most UIs should simply follow that sorting (instead of the current
> situation where each UI uses its own sorting).

The only problem is that completion-style doesn't have a point to
plug-in sorting yet.  So I thought good place to start is to place hints
in the completion-candidates.

> Still, in a case such as lisp/ecomplete.el where the completion table
> provides its own sorting based on recent-usage-frequency, how should
> this sorting be combined with that of flex?  I can see arguments in
> favor of saying that the flex-weight should be ignored in favor of the
> usage-frequency.

ecomplete has its own UI in ecomplete-display-matches? It's a question
of using or discarding the completion-style sorting hints stored in each
candidate.

My view of the matter is: completion table provides its sorting which
_can_ be overriden by completion style's sorting which _can_ be
overriden by completion UI's sorting.  AFAIU this can be done by writing
comparison functions that return nil if the current sorting should be
agnostic to both elements.

>>     Up until now, there was no way for completion styles to control
>>     sorting.  This change add such a facility and activated it for the new
>>     "flex" completion style.
>
> I'd like to move towards a completion-style API that uses cl-generic.
> E.g. introduce generic functions like `completion-style-try-completion`
> which would be like `completion-try-completion` but takes an additional
> `style` argument and dispatches based on it (replacing the dispatch
> based on completion-style-alist).

This is a good change, but it's not absolutely necessary to what I am
proposing right now.

>>     * lisp/minibuffer.el (minibuffer-completion-help): Use
>>     completion-style-sort-order and compeltion-style-annotation
>                                       ^^^^^^^^^^
>                                       completion
>>     properties.
>>     (completion-pcm--hilit-commonality): Propertize completion with
>>     'completion-pcm-commonality-score.
>>     (completion-flx-all-completions): Porpertize completion with
>                                         ^^^^^^^^^^
>                                         Propertize

Thanks.

>>     completion-style-sort-order and completion-style-annotation.
> Regarding annotations, I think being able to `concat` at the end is
> too limited.  If you take your example from sly, we'd like the "15%"
> annotations to be right-aligned and I don't see an easy way to do that
> with the facility you introduce.

I don't see the problem fully, but probably I'd rather remove the
annotation completely for now, until we have a better of how where we
want to place it.  As Dmitry suggested, the flex score annotation is
mostly a gimmick, we shouldn't add it until we have a good idea of how
it should work.
>
> The current annotation facility is also too limited in another way: in
> read-char-by-name we annotate each char name with the actual char it
> names, but we'd really like to prepend this annotation rather than
> append it (it would make the chars much easier to see and to compare),
> but the current annotation system doesn't offer this flexibility.
>
> So I'd encourage you to come up with a more flexible annotation system
> that allows prepending annotations as well as right-alignment (and
> ideally combinations of those, with several columns, ...).

What do you suggest?  I'd prefer small things that can be added
incrementally.  IOW, I don't have time for grand designs right now, I'm
mostly scratching the personal itch that is making icomplete.el not make
me miss ido.el so much.

So to summarize, I propose to add (1) the fafa9ec commit introducing
flex completion and (2) a new commit extracted from 2c7577558 which adds
the flex scoring calculation to each match, but doesn't do anything with
it yet.  Deal?

The first commit by itself can already greatly improve icomplete Ui.
And the second commit should encourage UI devs to start taking the
completion-style-sort-order hint into account.  If Dmitry does that in
company-capf, we get decent flex completion for little cost.

João