Re: [Emacs-diffs] emacs-25 f8208b6: Document the user-level features of the Xref package

[Eli Zaretskii]

> From: Dmitry Gutov <address@hidden>
> Date: Sun, 10 Jan 2016 06:02:42 +0300
> 
> Thanks for doing this.
> 
> I've noticed some problems, though, described below:

Thanks for reviewing the changes.

> On 01/09/2016 10:14 PM, Eli Zaretskii wrote:

>     +Emacs provides a unified user interface to these tools, called
>     address@hidden  The tools supported by @samp{xref} include:
>     +...
>     address@hidden GLOBAL, the source code tagging system, which provides
>     ...
>     +Cscope (@uref{http://cscope.sourceforge.net/}, a tool for browsing
>     ...
>     address@hidden IDUtils, a package for generating databases of
>     ...
>     +Grep, the venerable program that searches files for lines matching
>     ...
>     +Additional tools could be supported as they become available, or as
>     +user extensions.  Each such tool is used as a @dfn{backend} by
>     +commands described in this section.  Each command detects which
>     +backends are available for the current major mode, and uses the most
>     +capable of the available backends, with Grep generally serving as the
>     +fall-back backend.
> 
> Sorry, this is not true, unfortunately, as things currently stand.
> 
> We don't use GNU Global, Cscope, ID Utils, or Grep as xref backends. We can't 
> "find definitions" using any of them, for example, even though the user might 
> expect to be able to do so.
> 
> The only place where the aforementioned tools are used, is in the _default_ 
> implementation of the generic function xref-backend-references. IOW, the 
> implementation that gets used when the current backend doesn't provide its 
> own.

Sorry, I don't understand: AFAIK (and you seem to confirm) we do use
these tools in some situations, so the text is correct in saying that
they are used when appropriate.  So what is the nature of the problem
you have with this text?  Is it that I used "backend" in a meaning
that's different from what the xref implementation uses?  If so,
that's deliberate: this is a user manual, and that was the only
reasonable way I could think of to convey the idea of several backends
without talking about technicalities like xref-backend-references,
what symref does, when do we use the fallbacks, what are generic
functions, etc.

The idea the user will hopefully take out of this is that we use
several potential helpers under the hood some of which are based on
the mentioned tools that are not part of Emacs proper.  The tools are
mentioned to encourage the users to install them, so that their Emacs
environment is more complete and capable.

>     +The commands described here are useful for finding references in major
>     +modes other than those defined to support programming languages.  For
>     +example, chapters, sections, appendices, etc. of a text or a @TeX{}
>     +document can be treated as identifiers as well.  In this chapter, we
>     +collectively refer to a reference that specifies the name of the file
>     +where the corresponding subunit is defined, and the position of the
>     +subunit's definition in that file, as a @dfn{tag}.  We refer to the
>     +backends used by @code{xref} as @dfn{tagging backends}.
> 
> "tagging backends"? Why not call them "Xref backends", like referred to in 
> the Xref commentary and the docstrings?

I ended up not using this term, so I will just delete it, to avoid
confusion.

>     address@hidden Identifier Search
>     address@hidden Searching and Replacing with Identifiers
>     address@hidden search and replace in multiple files
>     address@hidden multiple-file search and replace
>     +
>     +  The commands in this section visit and search all the files listed
>     +in the @code{xref} backend's database, one by one.  For these
>     +commands, the database serves only to specify a sequence of files to
>     +search.
> 
> That may be true for tags-search, but that's never true for 
> xref-find-references.

It's true when xref-find-references uses IDUtils, for example.  And
even when it doesn't, the command does use _some_ database, even if
that database is made up of doc strings loaded into memory plus
load-history.

Once again, look at this from the user POV, not from the POV of
someone who knows how the implementation works.  Users need some
mental model of what's going on to make some sense and order out of a
heap of potentially unrelated features.  The above should provide
users with a conceptual framework that facilitates coming up with such
a mental model.  At least I hope it will.

> If the current backend defines xref-backend-references, then we just ask it, 
> and show the results.
> 
> If it doesn't, we delegate to the first available "CEDET tool", but they do 
> perform the regexp search, not just list the files. Then, 
> xref-collect-references takes the list of matches and verified, for each one, 
> whether the match begins and ends on a symbol boundary (Grep can't check 
> that).

I'm not describing xref alone, I'm describing what xref _and_ its
backend do together.  Differentiating between the two parts of the job
is not useful for the purposes of the user manual.

In any case, what you describe above is not something that can be
reasonably said in a user manual, can it?  You describe the (current)
implementation, whereas the user manual has quite different goals.
You are saying that the description I came up with loses some
accuracy, which is something I'm aware of, and which is inevitable
when describing complex software made up of several independent
modules whose features only partially overlap.

If you have suggestions for how to convey these ideas without going
into implementation, I'm all ears, of course.

>       These commands scan all the databases starting with the first
>     +one (if any) that describes the current file, proceed from there to
>     +the end of the list, and then scan from the beginning of the list
>     +until they have covered all the databases in the list.
> 
> What list? xref definitely uses one database at a time (per project root, if 
> we're talking about the default implementation).

Most of the commands described in the section from which this text
comes are in etags.el, for which the description is accurate.  The
only other command is M-? (xref-query-replace), for which this is
somewhat less accurate, but not entirely incorrect: a single database
is still a special case of "a list of all relevant databases".

Once again, feel free to suggest more useful ways of conveying the
ideas such that they make sense to users.  The above was the best I
could come up with, and I'll be happy to hear about possible
improvements, if they exist.

>     address@hidden tags-search
>     +  @kbd{M-x tags-search} reads a regexp using the minibuffer, then
>     +searches for matches in all the files in the selected tags table, one
>     +file at a time.  It displays the name of the file being searched so
>     +you can follow its progress.  As soon as it finds an occurrence,
>     address@hidden returns.  This command works only with the etags
>     +backend, and requires tags tables to be available (@pxref{Tags
>     +Tables}).
> 
> The last sentence makes it seem like it goes through the xref machinery, 
> whereas it doesn't.

The last sentence mentions backends because the parent section is
built upon that concept: that these commands use the appropriate
backend as they see fit.  That concept was introduced as an attempt to
make sense of the fact that xref implemented only part of what's
available with etags.  I do indeed think that IWBNI xref provided a
front-end for these features as well (it would have made my job of
documenting it a whole lot easier, if nothing else), but the text as
it stands stops short of saying explicitly that xref is involved.

OTOH, I don't see much harm if users somehow understand that xref is
involved (although the name of the command should tell otherwise).

>     address@hidden tags-loop-continue
>     +  Having found one match, you probably want to find all the rest.
>     +Type @kbd{M-x tags-loop-continue}) to resume the @code{tags-search},
>     +finding one more match.  This searches the rest of the current buffer,
>     +followed by the remaining files of the tags table.
> 
> This makes it seem like tags-loop-continue is applicable for use after 
> xref-find-* commands as well. But it isn't.

Well, it explicitly references tags-search and comes directly after
the description of that command.  Would it be better if the first
sentence said that explicitly, like below?

  Having found one match with @kbd{M-x tags-search}, you probably
  want to find all the rest.

> Maybe this command, as well as tags-query-replace, the other etags-only 
> commands, and the related etags variables, should still remain in a separate 
> section?

No, I don't think so.  It would make two sections out of one, which
would mean less order in the description.

>     address@hidden @kbd
>     address@hidden C-M-i
>     address@hidden address@hidden
>     +Perform completion on the text around point, using the @code{xref}
>     +backend if one is available (@code{completion-at-point}).
> 
> If I understand this sentence right, it's inaccurate: xref backends don't 
> define completion tables for buffer text. It is still governed by 
> completion-at-point-functions.

Is xref entirely not involved in completion-at-point-functions?  Do we
still use etags there?  The original text mentioned etags, and I
imagined that was replaced by xref now.  (I couldn't trace the code to
see what was it actually doing.)

>     @c Sadly, the new-and-improved Xref feature doesn't provide anything
>     @c close to the described below features of the now-obsoleted
>     @c tags-apropos.  I'm leaving this here to encourage enhancements to
>     @c xref.el.
> 
> Writing this complaint in the manual, expecting it to be read by someone 
> likely to produce enhancements to xref.el, seems rather optimistic.

I'm an optimist, yes.

> Please submit a feature request (or several, if they're distinct). Displaying 
> the tags files names should be possible after the xref rendering logic 
> becomes more flexible (and e.g. could be supplied by the backend).

Will do when I have time.

>       mode is enabled, it tries to use the Semantic parser data for
>       completion (@pxref{Semantic}).  If Semantic mode is not enabled or
>       fails at performing completion, it tries to complete using the
>     -selected tags table (@pxref{Tags}).  If in Emacs Lisp mode, it
>     +available @code{xref} backend (@pxref{Xref}).
> 
> Also not true. Like mentioned, we don't use xref backends for completion. 
> Unifying in-buffer completion and xref identifier completion might be 
> beneficial, but it's not easy to do.

I only changed the cross-reference, since there's no node named "Tags"
anymore; and etags is one of xref backends.  Are you saying that
completion-at-point has been changed not to use the tags table when
the Semantic mode is not enabled?  Or are you saying that this text
was incorrect even before my changes, and we didn't use tags tables
even before xref was added?

Thanks.