Re: Improvement proposals for `completing-read'

[Dmitry Gutov]

On 08.04.2021 11:37, Daniel Mendler wrote:

> > 6. would indeed make sense, and I'm not sure why we wouldn't want to 
> > have it in the non-selection case. Whatever come makes use of the 
> > completed value, could do the stripping of properties.
> >
> > Often, completing-read's caller can ensure the properties are there, 
> > by using something like (assoc-default completed-string collection) at 
> > the end. But that only works if the caller is also the provider of the 
> > completion table (otherwise it's an "opaque" data structure).
>
> Yes, you can use an alist and this is also what I use in my Consult 
> package when I want to obtain the associated data. But allowing 
> text-properties could simplify some scenarios and it would be more 
> natural when you are doing selection. If you consider proposal 7 
> (identity/deduplication), retaining the text properties is an automatic 
> outcome.

Perhaps. We might disable deduplication only optionally, though, OTOH 
retaining text properties seems to be always useful (even if caller 
wouldn't be able to always rely on it).

> > 9. completion tables need to be able to delegate all matching logic to 
> > an external process, both filtering and sorting. That's an important 
> > case for code completion, where we can take advantage of existing code 
> > and its "fuzzy matching" implementations.
>
> This would be neat, but it requires a lot of restructuring of the 
> completion logic. For this reason I am not fond of this idea. But you 
> can achieve something like this with your proposal 10. See what I 
> describe there, regarding Consult async.

AFAIK, being able to do this is essential to reach best performance in a 
number of important use cases.

> I tried to integrate `fzf` once with Consult async, like generating a 
> list outside Emacs, pushing it through `fzf` for fuzzy-filtering and 
> presenting it to the user via completion. But it turned out that most of 
> the external implementations are not good enough for this use case. They 
> don't have an option to open a pipe to update the filtering input for 
> example. I could write my own fuzzy matcher external backend which would 
> work perfectly with async completion. However then I can also just wait 
> for gccemacs :)

I was thinking more about interactions over network, with HTTP requests 
sent and received asynchronously. Mainly the cases where one uses the 
LSP protocol or similar.

> > 10. support for delayed/asynchronous fetching of completions which 
> > doesn't interrupt user's typing (it would generally abort the request 
> > if user input is detected, but there might be other approaches to 
> > that). Again, that's helpful when completions are produces by an 
> > external process.
>
> You may want to take a look at my Consult package, specifically the 
> async functionality. I believe that this functionality can easily be 
> provided on top of the current infrastructure, and actually in a nice way.

You can check out Company's asynchronous convention for backends:

https://github.com/company-mode/company-mode/blob/f3aacd77d0135c09227400fef45c54b717d33f2e/company.el#L456-L467

It's a very simple lambda-based future-like value. It can be updated to 
use a named type, and with other features too. I think it's a clean and 
simple base to build on, though.

> In Consult I am using closures which hold the asynchronously acquired 
> data. The closure function must accept a single argument, it can either 
> be a string (the new input) or it can be a list of newly obtained 
> candidates.

I'm not sure where to look, sorry.

> If it is a list the closure function must append the new candidates to 
> the already existing ones and return the full list of candidates. The 
> completion table then calls the async closure with either the input or 
> nil when doing all-completions.

I'm not 100% clear, but it sounds like chunked iteration. Which is a 
good feature to have. Though perhaps not always applicable to every UI 
(blinking with new results in a completion popup might be not 
user-friendly).

> Now a single problem remains - if new data is incoming the async data 
> source must somehow inform completion that new candidates are available. 
> In order to do this the async source must trigger the UI for example via 
> icomplete-exhibit/selectrum-exhibit and so on. It would be good to have 
> a common "completion-refresh" entry point for that. In Consult I have to 
> write a tiny bit of integration code for each supported completion system.

See my link, perhaps.

Or in general, a Future/Promise API has a way to subscribe to the 
value's update(s) (and the completion frontend can do that).

Having to use a global variable seems pretty inelegant in comparison.

> But since this proposal is much more complicated than the ones I didn't 
> add something like this here. Small steps first.

No hurry at all. Sometimes, though, a big feature like that can inform 
the whole design from the outset.