bug#40897: Negation of pixel-width :width expressions does not work

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Mon, 27 Apr 2020 08:47:50 +0000
> 
> In looking over the xdisp.c code, I noticed that
> calc_pixel_width_or_height has at least three problems:
> 
> 1. negation of width specifiers doesn't work

Thanks.  That should tell us how many people tried to use this feature
since it was introduced, 16 years ago ;-)

> 2. a circular list passed to (+) will hang Emacs
> 3. a circular list passed directly will crash Emacs
> 
> (1) is a significant bug and should be fixed. (2) is easy to fix, and
> we should probably do so on the master branch. (3) is difficult to
> fix, but I'd like to fix all these, and undiscovered similar issues,
> by converting calc_pixel_width_or_height to using the Lisp
> interpreter.

I don't think converting this to Lisp is a good idea, and I try to
explain below why.  It doesn't seem to be needed for fixing these
particular problems (see the proposed patch below), and even more so
given the evidence of this feature's popularity.

> The underlying problem is we're writing yet another mini-interpreter
> for a limited Lisp-like language, in C.

That is true, but there's a reason to this.

> That means no quitting, no protection against stack overflows

You cannot usefully quit or protect against stack overflow inside
redisplay anyway.  Quitting or signaling an error during redisplay
just gives you an endless redisplay loop, because signaling an error
immediately enters another redisplay cycle.  And even if you catch the
error and quietly return, in many/most cases you have another
redisplay either immediately or soon enough, and that presents a
locked-up or almost a locked-up Emacs ("almost" means that typing M-<
or M-> enough time might eventually get you out of the vicious circle,
if the problematic place is not at BOB/EOB).

The only reasonable way of avoiding these is to prevent the need to
error out.

You say (3) is difficult to fix, but the patch below does succeed in
preventing the stack overflow, so I don't think I understand why you
thought this to be difficult.  What am I missing?

> getting tricky semantics (such as the different behavior of (- A)
> and (- A B ...)) wrong.

AFAIU, the semantics is the same as in Lisp, isn't it?  The
implementation has a bug, but the semantics doesn't, I think.

> My proposal is to move the somewhat complex logic of
> calc_pixel_width_or_height to Lisp code, run when the display spec is
> actually evaluated (i.e. when "(when ...)" display specs run their
> Lisp code). This could be achieved by a method similar to the patch
> for bug#40845 in
> https://lists.gnu.org/archive/html/bug-gnu-emacs/2020-04/msg01096.html.
> 
> This would also resolve bug#40856 by allowing more complicated space
> calculations at a time when the font metrics are known.

You have promoted similar views elsewhere, on at least 2 other
occasions, so let me now try to explain why I very much object to
moving display functionality to Lisp, as a general tendency (as
opposed to some selected circumstances where this could be justified).

My reasons are as follows:

  . Lisp slows down redisplay.  First it does that because it's Lisp,
    and second because it causes GC.  We already have enough
    complaints against redisplay slowness (even on today's fast
    desktop machines!), so we should be very careful about this.

  . Handling Lisp errors inside redisplay is tricky, for the reasons
    explained above.  It is even more tricky when Lisp computes
    something that the rest of redisplay needs for its operation (as
    opposed to calling various hooks, such as window-scroll-functions,
    which are basically a one-way street -- if the hook errors out, it
    doesn't disrupt the display itself).

  . Last, but not least -- and people tend to forget about this
    important factor -- using Lisp to calculate display elements makes
    it nigh impossible to know when certain redisplay optimizations
    can be validly applied and when not.  That's because A Lisp
    program can potentially do anything and depend on any numbers of
    global and local state variables, and users will expect the
    display to change when these variables change their values.
    Redisplay optimizations are based on tracking some key state
    variables to deduce when a more thorough redisplay is needed and
    when it is safe to use shortcuts.  This is already a hard task,
    especially since we consistently made Emacs redisplay more and
    more lazily starting from Emacs 25 -- we still get bug reports
    about some situations where some state change fails to trigger
    redisplay, even though the number of possible variables to track
    is not very large.  Using Lisp much more than we do now will make
    tracking the relevant variables impossible.  The result will be
    only one: we will have to disable more and more optimizations and
    shortcuts to keep what's on the glass accurate.  And Emacs's
    redisplay can be *really* slow without the optimizations,
    especially in simple cases, like when you just move the cursor, or
    delete or insert a character.

So my suggestion is to use Lisp as part of redisplay only where no
other solution is reasonable or practical, and when the feature we
want is really important to have.

Once again, there could be certain situations where calling Lisp from
the display code might be the only practical solution for some serious
and important problem.  The above just tries to explain why I don't
think that idea is good _in_general_, and why we should try to avoid
it if at all possible.

I hope I made my position on this clear enough.

Here's the patch I came up with to handle the problems you reported.
WDYT?

diff --git a/src/xdisp.c b/src/xdisp.c
index 140d134572..bc27fb15e0 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -27284,11 +27284,16 @@ calc_pixel_width_or_height (double *res, struct it 
*it, Lisp_Object prop,
              pixels = 0;
              while (CONSP (cdr))
                {
+                 if (EQ (cdr, XCDR (cdr)))
+                   return false;
                  if (!calc_pixel_width_or_height (&px, it, XCAR (cdr),
                                                   font, width_p, align_to))
                    return false;
                  if (first)
-                   pixels = (EQ (car, Qplus) ? px : -px), first = false;
+                   pixels =
+                     (EQ (car, Qminus) && CONSP (XCDR (cdr))
+                      ? -px
+                      : px), first = false;
                  else
                    pixels += px;
                  cdr = XCDR (cdr);
@@ -27305,13 +27310,15 @@ calc_pixel_width_or_height (double *res, struct it 
*it, Lisp_Object prop,
 
       /* '(NUM)': absolute number of pixels.  */
       if (NUMBERP (car))
-{
+       {
          double fact;
          int offset =
            width_p && align_to && *align_to < 0 ? it->lnum_pixel_width : 0;
          pixels = XFLOATINT (car);
          if (NILP (cdr))
            return OK_PIXELS (pixels + offset);
+         if (EQ (cdr, XCDR (cdr)))
+           return false;
          if (calc_pixel_width_or_height (&fact, it, cdr,
                                          font, width_p, align_to))
            return OK_PIXELS (pixels * fact + offset);