bug#31717: 26.1; display-line-numbers-mode enlarges indicator without need

[Eli Zaretskii]

tags 31717 notabug
thanks

> From: Carlos Pita <carlosjosepita@gmail.com>
> Date: Tue, 5 Jun 2018 00:30:10 -0300

(Sorry for a longish response, but you raise design and implementation
issues which cannot be explained without some non-trivial background.)

> When transitioning from line 69 to line 70 a new column is added to the
> left of the indicator as if there was a need to fit three digits despite
> that the file has less than 100 lines and despite that if it were larger
> line 100 would fall out of the window.

Right, this is the intended behavior, and there are good reasons for
it.  See below.

> It's as if the mode is anticipating an extra digit soon and playing
> conservative. I understand that there could be performance reasons for
> doing this (thus avoiding the need to compute the remaining lines in a
> window or file)

The main reason is performance, but it's not the only one.  You must
keep in mind that the code which produces line numbers is part of the
display engine -- it produces the line number as part of that line's
layout -- and that imposes very specific restrictions and conditions
on what we can do, how, and when.  I try to explain the issues below
in more detail, but the main point is that you simply _cannot_ think
about stuff done in redisplay as if it were some Lisp program running
off post-command-hook or somesuch, it's a very different runtime
environment with peculiar requirements.

> but:
> 
> 1. The mode is already adding two whitespaces around the line number
> indicator. Wasting one more char is on the verge of infuriating ;)

If you don't want to "waste" one more column, you can use one of the
optional features, e.g., set display-line-numbers-width-start non-nil,
with or without display-line-numbers-grow-only non-nil.

> 2. AFAICS computing the file size doesn't seem too demanding a task,
> neither the remaining lines in a window  (although the window could be
> resized).

Doing this as part of a redisplay cycle is precisely what made linum
mode so slow.  In a nutshell, it would require displaying each window
twice, because the exact number of lines in a window is known only
after it was completely redrawn (remember that Emacs supports
variable-size fonts and non-character display elements such as images,
which all affect the number of lines in a window).

As for computing the number of lines in a buffer, I guess you never
have to deal with very large buffers, if you think it's scalable.
With today's 64-bit architecture, the maximum buffer size supported by
Emacs is too huge to allow us to count lines all the time; plus, as I
explain below, the display code which produces line numbers is called
much more than you evidently assume.

> Also, when the mode is visual or relative this is happening too.

By default, visual and relative line numbers still display the
absolute line number for the current line, so we still need enough
space for that.  And the current line could be the last line of the
window.  (And if you think the fact that only the current line needs
this makes the life of the implementation easier, read on, and you
will understand why not.)

> In these cases, given that I assume you are assuming <= 69 is a safe
> guess, it will be *always* safe to limit the indicator width to that
> required by the current line.

As you later discovered, there's no 69 value anywhere in the code, the
space needed for the line-number display is estimated dynamically and
depends on the window size and the font size used by the window's
frame.  For a very large window, the switch could be much sooner than
line 69; for a very small window, it could be very close to 100.

> A little more experimentation with different windows sizes made me
> realize that the <70 hard and fast rule doesn't exist. Now I see the
> threshold indeed depends on the window size. But it is still too
> conservative, leading to premature enlargement of the line number
> space.

It _must_ be conservative, because it needs to decide on the space for
the line numbers _before_ it displays the first line of the window.

Here's some background information regarding the requirements imposed
by the display engine on the line-number implementation:

The Emacs display engine works by "screen lines".  (A screen line is a
single horizontal line of the display "canvas"; continuation lines
count as additional screen lines.)  The workhorse of the display
engine is a function that lays out a single screen line.  In some
cases Emacs invokes that function to redraw all the lines of a window,
one by one; in other cases, the function is invoked to produce only a
few screen lines, or even just one, when Emacs is able to decide up
front that none of the other screen lines need to change on display --
being able to do this is an important part of Emacs redisplay
optimizations.

In addition, Emacs needs to take text layout on display into
consideration in many situations that have little to do with actual
redisplay.  For example, vertical-motion (which is the basis of any
vertical cursor movement and scrolling commands) needs to determine
the character directly below or above the character at point,
something that obviously depends on the stuff displayed between point
and that other place, the faces involved, etc.  As another example,
consider a mouse click inside the window, where Emacs needs to know
the buffer position at the click coordinates.

For these reasons, the display routines that perform layout
calculations are used _a_lot_ in Emacs, and must (a) be very
efficient, and (b) produce consistent results no matter whether they
are called to redraw the entire window or just a small part thereof.
In particular, the functions that lay out a single screen line have no
idea about the context in which they were called -- they don't know
whether they are called as part of a complete window redisplay, or
just to draw a single screen line.  They must produce exactly the same
results regardless, and without depending on any such context.

I hope you are now beginning to understand why the feature works as it
does.  The code must calculate the space required for line-number
display before it displays the first line of a window, and the result
must be guaranteed to fit for all the lines in that window, because
otherwise some window line near the end might need more space, and you
will either have an ugly "jagged" display, or will need to abandon the
redisplay cycle and trigger another cycle -- which will be slow
(that's basically what linum.el does, and we already know it's slow).
How do we estimate the largest line number to be displayed in the
window in a way that is never too small?  Well, the display engine
keeps an estimation of the maximum number of screen lines in the
window, based on the fonts defined by the frame -- it needs to know
that because that defines the dimensions of the canvas used for
displaying the window.  So we simply reuse that estimation for this
purpose.  That estimation is always a conservative one, and it cannot
be any other way.  (Btw, it is much less conservative on TTY frames,
for obvious reasons, so there you will see much less "waste".)

> I suggest three optimizations in order of assumed (wild guessed)
> ascending difficulty:
> 
> 1. For relative/visual modes never reserve extra space.

Cannot be done, because when the space for line numbers is computed,
Emacs doesn't yet know what will be the current line, nor where in the
window it will be.  The current line, which is the line that displays
the cursor, is computed only near the end of the redisplay cycle of a
window.  And the code must use the same conservative estimate to be
able to display the largest line number possible.  If you don't need
to see the absolute line number in relative mode, customize
display-line-numbers-current-absolute to nil, and the "waste" will
indeed be no more.

> 2. If the lines in the file are < N never reserve extra space for >= N.

Can't be done by default, because counting lines in a very large
buffer will slow down redisplay and any command that uses display code
(see above).  It would also cause unpleasant redrawing and horizontal
movement of text when you add lines at the end of the buffer.

However, you can have this as an optional feature, if you customize
display-line-numbers-width-start to a non-nil value.

> 3. Take also into account the remaining window lines (I assume this is
> not as easy as it sounds).

Can't be done, because it would require a second redisplay cycle,
which will get us back to the same performance problem as we have in
linum.el.  I didn't write this feature just to have it work as slow as
linum.el, only in C ;-)

(Of course, if someone has clever ideas how to improve line-number
display, please speak up, but you should make yourself familiar with
the related code in the display engine first.)

I hope the above explains why we have the implementation we have.  All
in all, I find the price entirely reasonable, given the performance
gains (the default configuration works with line numbers almost as
fast as without them).  And we have a few customizable options to
optionally enable features which alleviate some of these issues, for
those who are willing to pay the price.

P.S.  Please make a point of keeping the full title of the bug in your
responses and followups, because removing the title and leaving just
the bug number makes it much harder to find all the messages that
pertain to the same bug, at least with Rmail.

Thanks.