bug#11860: 24.1; Arabic - Harakat (diacritics, short vowels) don't appear

[Eli Zaretskii]

> From: Kenichi Handa <handa@gnu.org>
> Cc: jasonr@gnu.org, 11860@debbugs.gnu.org, smias@yandex.ru
> Date: Tue, 21 Aug 2012 22:16:51 +0900
> 
> > (Note that the fRTL member of items[0].a is set to TRUE.)  My
> > understanding of the advances[] array is that it gives, for each glyph
> > in the cluster, the number of pixels to advance to the right after
> > drawing the glyph.  So the fact that it is 8 for the first (base)
> > character and zero for the second one tells me that this grapheme
> > cluster is supposed to be rendered in reverse order: first the Sukun,
> > then Ayin at the same location, and then advance by 8 pixels for the
> > next character.  Is this correct?
> 
> I think so.

Well, it turns out that the truth is slightly different.  When the
Uniscribe shaper is handed a chunk of RTL text with the fLogicalOrder
flag set to TRUE, it prepares the glyphs in the logical order, but
assumes that they will be laid out in reverse.  In this reverse order,
the width advance value is applied _before_ drawing the glyph, and
positive width advance values move the pen to the _left_.  I found
this important information on some Web page, which I unfortunately can
no longer find.

In addition, it looks like in this "reverse" mode, the X-OFFSET value
is also interpreted in the reverse direction, so its sign must be
flipped for glyphs in RTL grapheme clusters.

Armed with this knowledge, with the information you posted, and after
studying the drawing code in w32term.c, I made some semi-empirical
changes in uniscribe_shape that produce good results both with Arabic
and with Hebrew.  In a nutshell, I adjusted the X-OFFSET values for
the width of the base-character glyph.  The results are committed as
trunk revision 109726; as I only tested the modification on a small
sample of composed texts, please see if you can run more tests with as
complex compositions as you can find.

> > If it is correct, then how come the glyphs shown on GNU/Linux also
> > have non-zero value of xadvance:
> 
> >   [0 1 1593 969 8 2 8 4 4 nil]
> >   [0 1 1618 760 0 -6 -3 8 -11 [-9 2 0]]
> 
> Emacs draws the first glyph at its base point and advance
> the base point 8 pixels to the right (because the WIDTH of
> the first glyph is 8).  Then Emacs draw the second glyph at
> 9 pixels left and 2 pixels up from the base point.  So, the
> second glyph is drawn above the first glyph.

I see.  This was somewhat counter-intuitive (why move first and then
correct that by negative offsets, instead of not moving until all the
glyphs in the cluster are drawn?).

> > > For instance, in the above case, we may have to render glyphs in
> > > this order (diacritical mark first):
> > > 
> > >   [0 1 1593 760 0 3 6 12 4 [1 -2 0]]
> > >   [0 1 1593 969 8 1 8 12 4 nil]
> 
> > I tried the naive patch below, but it didn't quite work.  It seems
> > like those changes somehow prevented character composition.  Perhaps
> > Handa-san could give me some guidance here.
> 
> Did your patch produced the above GSTRING?

Yes.  But I think swapping the glyphs in the cluster was not the right
idea, because it violates the assumptions in w32font_draw, the drawing
routine called by the font back-end.  That routine expects the first
glyph to be for the base character of the composition.

> Please see the function
> x_draw_composite_glyph_string_foreground (in xterm.c and
> w32term.c).  It shows which component of GSTRING is used for
> drawing (the last branch of "iff" condition).

Yes, that helped, thanks.

Steffan, can you try the latest trunk code, and see if there are any
problems left?