bug#70541: track-changes-mode logs warnings (with input method, in Eglot buffer)

[Stefan Monnier]

>> >> The buffer state is modified by Quail.  It's somewhat temporary but
>> >> there's still a lot that can happen during that temporary state.
>> > It isn't just temporary: it's a change that "isn't supposed to exist".
>> > It's just a side effect of how Quail is implemented.
>> 
>> But what does it mean concretely?  In which sense is it supposed not
>> to exist?
>> And more to the point, what makes it important to hide those changes?
>
> The actual change is the character(s) inserted when the Quail input
> function completes its job.  Everything else is ephemeral, and Quail
> deletes it as part of its job.

While we can think of it as ephemeral, the fact is that it lasts through
redisplay, running timers, etc...

So it's not at all obvious why it should be treated differently from the
case where the users type something and then hit undo because they
realize they made a mistake, or when they type a short string followed
by `M-x expand-abbrev`.

There are many other forms of "ephemeral" changes, where we don't
inhibit modification hooks.  E.g. in `replace-match` we sometimes insert
the replacement and then capitalize it.  The intermediate
non-capitalized string is very much ephemeral: much more ephemeral than
Quail's (there's *very* little code run between the two operations).
Yet we call `before/after-change-functions` separately for both changes,
thereby exposing the intermediate state.

> So those insertions followed by deletions are not meant to change the
> buffer,

Yet they do.

> and should therefore be ignored.

I don't understand the "therefore".

Also, ignored by what?
Clearly they should not be ignored by redisplay (since the whole
purpose is to give guidance to the users).

Also, currently they're not ignored by `font-lock`.  Is that a bug?

>> It also works correctly with the new code.  The difference is that we
>> report it (notice the `Subject:` says "warning").
>> [ Note also that `track-changes.el` does not warn about it when running
>>   in a released version of Emacs (see `track-changes-record-errors`),
>>   because I assume it's less useful.  ]
> So this is actually a non-issue?

It's a performance bug.

>> The change are there.  `point`, `point-max`, `current-column`,
>> etc... are affected.
> Why does Eglot need to react to those changes immediately when they
> happen?

It doesn't: it reacts to the changes in an idle timer.

But the idle timer can see Quail's intermediate state, exactly because
it is not nearly as ephemeral as would be needed for
`with-silent-modifications` to be acceptable.

> So I'd very much prefer that Quail signaled to applications that it's
> in the middle of handling some complex input, and that applications
> which track changes ignored the changes made during this period.

Eglot could pause its work during Quail input.  But note that with some
input methods, being in the middle of a Quail input can be very
common, so it's not a great solution because it could unduly delay the
work of Eglot.

And of course, other packages may not have that luxury.

> We might already have variables in Quail which could be used as such
> flags: quail-translating, quail-converting, quail-current-str, and
> quail-guidance-str, to name a few candidates.  Could any of them be
> used for this purpose?

I definitely would not want Track-Changes to lookup a Quail variable.
We should instead use/introduce some package-neutral var (which other
packages could use).  But I'd first like to know why it is that Quail
needs `inhibit-modification-hooks` to make sure we really need such
a workaround.
[ BTW, let-binding some dynbound variable would not do since
  track-changes would not see it if called from a separate thread.
  It would have to be a buffer-local var.  ]


        Stefan