bug#20847: [display engine] 25.0.50; company-mode popup makes point jump to an entirely different location

[Eli Zaretskii]

> Cc: 20847@debbugs.gnu.org
> From: Dmitry Gutov <dgutov@yandex.ru>
> Date: Wed, 24 Jun 2015 00:15:57 +0300
> 
> On 06/23/2015 10:07 PM, Eli Zaretskii wrote:
> 
> > "Now" as in "what Company does now".
> 
> It very much uses that newline "now".

Yes, and I was asking whether in this specific case it could refrain
from doing that.  (And yes, I understand that doing so would be
additional complexity for you.)

> > emacs -Q
> > M-: (overlay-put (make-overlay 65 66) 'before-string "how\ndo\nyou\ndo?") 
> > RET
> 
> But there's nothing special about newlines here. The display engine 
> probably never "tried" to display the cursor at any of those that are 
> inside the display string.

Yes, it did.  When the display engine needs to decide where to put the
cursor, it examines all the screen lines whose buffer positions are
around point.  So in this case, it did examine all those lines that
came from an overlay string, and rejected them all.

> >> The fact that you'd *try to* display the cursor at the newline
> >> belonging to an overlay display string indicates that the overlay
> >> must start at that position, doesn't it? Or end.
> >
> > Sorry, I don't follow: what do you mean by "you'd try"?
> 
> Consider for displaying the cursor at?
> 
> Allow me to quote yourself: "In a nutshell, when a screen line ends in a 
> newline that comes from an overlay string, we don't want to display the 
> cursor on that line."
> 
> For that heuristic to apply, I'd say first you have to "try" to display 
> it at that position.

We do, see above.  We actually examine all those lines one by one.

> >> If it starts earlier, then the cursor might be displayed before or after
> >
> > We always display it after, when point is at buffer position that has
> > an overlay string property.
> 
> So maybe I misunderstood, and there's no heuristic about newline at the 
> end of visual line coming from an overlay.

No, you didn't misunderstand.  There is such a heuristic.

> And that other circumstances conspire to make it seem that way,
> namely:
> 
> - Cursor is always displayed after an overlay when it's "inside" it.
> - We somehow always consider point to be "inside" overlay when it's at 
> the window edge, and an overlay begins there too.

That's correct, but the heuristic applies to the line where the
overlay string ends, when it ends with a newline that comes from the
overlay string.  When this happens, we could place the cursor either
at the end of the line with that newline, or at the beginning of the
next line.  We prefer the latter.

> Maybe the latter could still be changed.

I tried to do that, but unfortunately, when the display engine gets to
the point where it needs to place the cursor, it lacks information for
treating this situation specially.  That's because the information
about the overlay is long gone, and the newline didn't leave any
glyphs on the screen.  So it's impossible to discern between this
situation and any other where we have a a line ending in a newline
that comes from an overlay string, without once again searching for
the overlays in the buffer, soring them, finding the ones that should
have been displayed, etc.

> >> If it ends later, why would we even try to display the cursor at
> >> that newline in the first place?
> >
> > Because its corresponding buffer position matches point.  That's how
> > cursor positioning works in Emacs: it always starts by looking for the
> > glyph on the screen that corresponds to the buffer position where
> > point is.  If such a glyph is not found, for one of the many reasons
> > (invisible text, overlay or display string, hscroll, you name it),
> > then the fun begins: we try very hard to find some alternative
> > position that fits.
> 
> Then the overlay must begin at the edge of the window, and all the cases 
> where your quote (above) applies look like this case.

Yes.  And that's exactly the problem: we would like to treat this case
differently, but don't have the necessary information to do so.

What I need to do so is (1) the starting and ending point of the
overlay which begot the string, and (2) the fact that the string
begins with a newline, or some handle for the string itself.  None of
that is available at the point where we decide whether this line is
"appropriate" for the cursor.

> >> But then, maybe I don't really understand what it does.
> >
> > I can tell you which parts of code to read, if you want.
> 
> If you think that's wise.

It's not up to me, it's up to you.  If you want to understand the gory
details without myself standing in between (and maybe describing some
things inaccurately or not clearly enough), then I will gladly tell
you where to look.  I'm also okay with describing it for you as best I
can, like I did until now.  It's your call.