anomalies of overlays and before-string and display properties

[Joe Wells]

There are a number of strange anomalies (some clearly bugs) in the
behavior of overlays and before-string and display properties.

1. Faces on a display property on a before-string behave differently
   depending on where the display property is in the before-string.
   For a display property at the beginning of the before-string, its
   face is used and the before-string's face is ignored (I think this
   is correct).  For a display property not at the beginning of the
   before-string, its face is completely ignored, and the face of the
   before-string is used instead.  This is fairly clearly a bug.

2. When a substring of a before-string beginning at offset O has a
   display property which is a string S, the first O characters of S
   are not displayed.  If S is not at least O+1 characters long, then
   disastrous things start to happen.  This is clearly a bug.

3. The before-string somehow “inherits” face attributes from the face
   of the character that follows the start of the overlay.  (If the
   overlay is of length zero, this character is not even in the
   overlay.  See the recent discussion of how this problem is
   affecting linum.el.)  This means that in practice a before-string
   needs to use completely specified faces, which is obviously a real
   pain.

4. The display property of an overlay somehow “inherits” face
   attributes from the face of the character that follows the start of
   the overlay.  This has similar disadvantages to anomaly #3
   mentioned above.

5. The help-echo and mouse-face properties of an overlay with both a
   before-string and a display property only affect the material
   displayed by the overlay's display property.  One can work around
   the issue by copying these properties to the various display
   properties inside the before-string; however, this workaround is a
   pain and in the case of mouse-face it doesn't get the entire
   overlay highlighted at once but instead only the pieces are
   highlighted one at a time.  It would be nice to be able to set a
   single help-echo or a single mouse-face property that would affect
   the _entire_ overlay.

You can reproduce all of these anomalies with this function:

  (defun illustrate-anomalies ()
    (let ((s #1=#("\\abcd{VWXYZ}"
                  0 5 (face (:foreground "Purple" :background "yellow"))
                  5 12
                  (face
                   (:foreground
                    "DarkOliveGreen"
                    :weight bold :slant oblique :height 0.8))))
          (props
           '(help-echo
             #1#
             ;;face (:slant italic :foreground "red")
             display "]"
             before-string
             #("  "
               0 1
               (display
                #("|["
                  0 2 (;; help-echo #1#
                       face #2=(:strike-through t :background "white")))
                ;; help-echo #1#
                face #3=(:underline t))
               1 2
               (display
                #("||VWXYZ"
                  0 7 (;; help-echo #1#
                       ;; mouse-face (:background "darkseagreen2")
                       face #2#))
                ;; help-echo #1#
                face #3#))
             mouse-face (:background "darkseagreen2")))
          (buf (get-buffer-create "xyzzy")))
      (with-current-buffer buf
        (display-buffer buf)
        (erase-buffer)
        (dolist (o (overlays-in (point-min) (point-max)))
          (delete-overlay o))
        (insert s)
        (let ((o (make-overlay (point-min) (point-max))))
          (while props
            (overlay-put o (car props) (cadr props))
            (setq props (nthcdr 2 props)))))))

This example is derived from some work where I was trying to get
AUCTeX's folding mode to show the fontified and latex-previewed
contents of macro arguments.

I hope this problem report is useful.

Joe

======================================================================
In GNU Emacs 22.1.1 (i686-pc-linux-gnu, GTK+ Version 2.8.20)
 of 2007-06-27 on artemis
Windowing system distributor `The X.Org Foundation', version 11.0.70000000
configured using `configure  '--prefix=/home/jbw/local2' '--enable-debug' 
'--disable-nls' '--with-x-toolkit=gtk' 'CFLAGS=-O0 -g3 -ggdb''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: jbw
  value of $LANG: nil
  locale-coding-system: utf-8
  default-enable-multibyte-characters: t

Minor modes in effect:
  outline-minor-mode: t
  desktop-save-mode: t
  url-handler-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  size-indication-mode: t
  line-number-mode: t
  transient-mark-mode: t