bug#34150: 26.1; Document filtering with `isearch-filter-predicate' in Elisp manual

[Drew Adams]

> > > This is not a user-level facility, so the user perspective is not
> > > relevant.
> >
> > The "user perspective" is always relevant for Emacs.
> > We are all users.
> >
> > Searching is a user-level facility.  If a search
> > imposes/provides filtering, that certainly affects
> > user-visible behavior.  Users should understand
> > that behavior.
> >
> > Anyone imposing such filtering on users should
> > consider its effects on users, including the
> > considerations raised by this bug report.  That
> > starts with making the behavior and consequences
> > clear to filter implementors.
> 
> All true, but the filtering itself is not on the user level: the
> _results_ of the filtering are.  So the behavior on the user level
> should be described and understood by users in higher-level terms.

Correct.  It should be.

But those who implement filter predicates also need
to be aware of the behavior, and to think in user
terms, in order to think of documenting it for their
users.

I know, speaking as one such implementor.  When the
resulting filtering behavior does not have an obvious
explanation without understanding something about
filtering, an explanation for users is called for.

> For example, with the default filtering, the behavior should be
> described in terms of searching inside invisible text, not in terms of
> filtering out some of the hits; the latter is just the implementation,
> not the user-level behavior.

Uh, no, aside from Isearch being able to "open" hidden
text containing matches - which is something else again,
use of such filtering by Isearch is not about searching
inside INvisible text.  It's about filtering out some
of what would otherwise be considered search hits.

Aside from the exceptional behavior of being able to
"open" invisible text, search with filtering is about
search visible, not invisible, text.

Not to mention that users can, themselves, add filters
to exclude searchable text.  (With my Isearch+ code
they can even do that on the fly, interactively.)

Thinking in terms of filtering, excluding possible
matches is entirely appropriate at the user level.

It's akin to narrowing a set of completion candidates
by progressively imposing additional match constraints.

And even for completion there are two possibilities
of filtering candidates: before matching user input
or afterward.  There are specific advantages to each.
And yes, when one or the other is employed (or both)
it can be important for users to know this, as it can
affect not only what they see but how they choose to
interact with the completion UI.

The grain of truth in what you say is that in some
cases users need not be aware of _some_ filtering
that goes on.

And even in those cases the possibility of their
ignorance can depend on the kind of matching
employed.  See my example of regexp matching within
a rectangle - no such problem for simple string
matching.

For simple, non-regexp searching users generally
need not think in terms of (1) searching the whole
buffer for matches and then (2) excluding matches
that extend beyond the visible text.

In that case a user-level description wouldn't
need to mention filtering at all.  Not so for the
regexp-search case, however.  Users need some
description/explanation to provide them a conceptual
model that explains the behavior they'll run into. 

> Therefore, the details of how filtering of matches works, and how to
> write a filter, is not user-level information.

Certainly user-visible behavior should be described
to users in user terms, not implementation terms.
The implementation is irrelevant to them (unless
they dig into Lisp during interaction).

But a user description here must cover what I said,
one way or another.  They need to know why their
regexp of .* does not find a match in the visible
text, when they can see a match for it.

Their conceptual model of what's happening, what's
going on, needs to make this clear to them.  It need
_not_ be in terms that mirror the implementation.
But it needs to describe/explain the behavior they
see, one way or another.

> If you re-read what
> you wrote above, you will see that your arguments are all consistent
> with what I said, they don;'t contradict that in any way.

Dunno what expected contradictions you want me to
look for.  I would like to see doc that makes users
aware of the behavior they run into.  Today that's
not the case.

And a separation of programmer-level from user-level
doc does not preclude making programmers themselves
aware of what behavior users need to expect or know
about.  If they don't themselves understand the
behavior then they can't manage it well or explain
it to their users.