bug#35811: 27.0.50; Arabic character (de)compositions affected by edits elsewhere in buffer

[Basil L. Contovounesios]

This report is a followup to bug#35721[1] focussing only on the
alternating composition of Arabic characters when editing seemingly
unrelated parts of the buffer.

[1]: https://debbugs.gnu.org/35721

Observe:

0. emacs -Q
1. C-u C-\ arabic RET
2. a ; C-a C-u C-x =

--8<---------------cut here---------------start------------->8---
             position: 146 of 147 (99%), column: 0
            character: ش‎ (displayed as ش‎) (codepoint 1588, #o3064, #x634)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x0634
               script: arabic
               syntax: w        which means: word
             category: .:Base, R:Right-to-left (strong), b:Arabic
             to input: type "a" with arabic input method
          buffer code: #xD8 #xB4
            file code: #xD8 #xB4 (encoded by coding system utf-8-unix)
              display: composed to form "ش" (see below)

Composed using this font:
  xft:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-15-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 0 0 3186 9 -1 9 9 1 nil]

Character code properties: customize what to show
  name: ARABIC LETTER SHEEN
  general-category: Lo (Letter, Other)
  decomposition: (1588) ('ش')

There are text properties here:
  fontified            nil
--8<---------------cut here---------------end--------------->8---

3. C-e RET

The sheen is correctly shaped in its initial form:

3. a ; RET

The first sheen unexpectedly changes to its isolated form:

4. a

The first sheen reverts to its initial form:

5. ; RET 

Now the second line of Arabic is decomposed.

6. C-p C-p C-a C-u C-x =

Now the second line of Arabic is composed again.

--8<---------------cut here---------------start------------->8---
             position: 149 of 154 (96%), column: 0
            character: ش‎ (displayed as ش‎) (codepoint 1588, #o3064, #x634)
              charset: unicode (Unicode (ISO10646))
code point in charset: 0x0634
               script: arabic
               syntax: w        which means: word
             category: .:Base, R:Right-to-left (strong), b:Arabic
             to input: type "a" with arabic input method
          buffer code: #xD8 #xB4
            file code: #xD8 #xB4 (encoded by coding system utf-8-unix)
              display: composed to form "ش" (see below)

Composed using this font:
  xft:-PfEd-DejaVu Sans Mono-normal-normal-normal-*-15-*-*-*-m-0-iso10646-1
by these glyphs:
  [0 0 0 3186 9 -1 9 9 1 nil]

Character code properties: customize what to show
  name: ARABIC LETTER SHEEN
  general-category: Lo (Letter, Other)
  decomposition: (1588) ('ش')

There are text properties here:
  fontified            t
--8<---------------cut here---------------end--------------->8---

Notice fontified is t now.  I don't think this matters much (because
there doesn't seem to be a correlation between character decompositions
and the value of this property), but could font-lock or some other major
mode feature have something to do with this issue?

0. emacs -Q
1. DEL                [optional, forces L2R paragraph direction]
2. M-x text-mode RET  [fundamental-mode also works]
3. C-u C-\ arabic RET
4. a ; RET a ; RET

Sure enough, the letters never decompose.

Note that, in the lisp-interaction-mode examples, the characters on
previous lines decompose not only when inserting repetitions of "a ;
RET", but also when deleting these insertions with repetitions of DEL.

Details of the three Emacs versions I can reproduce this on (master,
harfbuzz, emacs-26) follow my signature.

Thanks,

-- 
Basil

In GNU Emacs 27.0.50 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll 
bars)
 of 2019-05-20 built on thunk
Repository revision: afdc20d73c8588e5a744ecf7bffaf4401a557d20
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux 10 (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O2 -march=native' --config-cache
 --prefix=/home/blc/.local --with-mailutils --with-x-toolkit=lucid
 --with-modules --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG SOUND GPM DBUS GSETTINGS GLIB NOTIFY
INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT ZLIB
TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD JSON
PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_IE.UTF-8
  locale-coding-system: utf-8-unix


In GNU Emacs 27.0.50 (build 2, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll 
bars)
 of 2019-05-13 built on thunk
Repository revision: 5d7dafacf4afc888511649f6fc24c28210cd0dfc
Repository branch: harfbuzz
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description: Debian GNU/Linux 10 (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4'
 --config-cache --prefix=/home/blc/.local --program-suffix=-harfbuzz
 --enable-checking=yes,glyphs --enable-check-lisp-object-type
 --with-mailutils --with-x-toolkit=lucid --with-modules
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY INOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE HARFBUZZ
M17N_FLT LIBOTF XFT ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES
THREADS LIBSYSTEMD JSON PDUMPER LCMS2 GMP


In GNU Emacs 26.2.50 (build 5, x86_64-pc-linux-gnu, X toolkit, Xaw3d scroll 
bars)
 of 2019-05-20 built on thunk
Repository revision: 122ba1689046c53535b4d6c5142cfd81752808d0
Windowing system distributor 'The X.Org Foundation', version 11.0.12003000
System Description:     Debian GNU/Linux 10 (buster)

Configured using:
 'configure 'CC=ccache gcc' 'CFLAGS=-O0 -g3 -ggdb -gdwarf-4'
 --config-cache --prefix=/home/blc/.local --program-suffix=26
 --enable-checking=yes,glyphs --enable-check-lisp-object-type
 --with-mailutils --with-x-toolkit=lucid --with-modules
 --with-file-notification=yes --with-x'

Configured features:
XAW3D XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK SOUND GPM DBUS GSETTINGS
GLIB NOTIFY ACL LIBSELINUX GNUTLS LIBXML2 FREETYPE M17N_FLT LIBOTF XFT
ZLIB TOOLKIT_SCROLL_BARS LUCID X11 XDBE XIM MODULES THREADS LIBSYSTEMD
LCMS2