bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n)

[Eli Zaretskii]

> Date: Sat, 5 Jan 2019 22:53:14 +0200
> From: Khaled Hosny <dr.khaled.hosny@gmail.com>
> Cc: far.nasiri.m@gmail.com, behdad@behdad.org, 33729@debbugs.gnu.org
> 
> > Done.  Please test.  I made sure it compiles, but I couldn't actually
> > test the results, as I don't have access to a GNU/Linux system with
> > GUI display.  So it could be that I misunderstood the Harfbuzz APIs,
> > as I was essentially flying blind, guided only by the Harfbuzz docs.
> 
> It seems to work, but still not quite right. You seem to be passing the
> paragraph direction, but what HarfBuzz needs is resolved direction of
> the text (i.e. the bidi embedding level of the run).

It isn't the paragraph direction; at least it wasn't supposed to be
that.  The code is (or was before your changes):

      if (charpos < endpos)
        {
          if (pdir == L2R)
            direction = QL2R;
          else if (pdir == R2L)
            direction = QR2L;
          [...]
          cmp_it->reversed_p = 0;
        }
      else
        {
          [...]
          cmp_it->reversed_p = 1;
          [...]
          if (pdir == L2R)
            direction = QR2L;
          else if (pdir == R2L)
            direction = QL2R;
          [...]
        }

So, as you see, when the paragraph direction is L2R, normal text gets
L2R direction, while test reversed for display gets R2L, and the other
way around when the paragraph direction is R2L.  Which AFAIU is what
HarfBuzz needs, but maybe I'm missing something.

Did you actually see incorrect display with the code I wrote?  If so,
could you please show the recipes for reproducing that, preferably
with screenshots of correct and incorrect display?  I'd like to look
into that, to understand what I missed.

> HarfBuzz direction guessing should never be used (i.e.  always pass
> to it an explicit direction).

This is in general impossible (or at least very hard), since the
shaper is sometimes called from Lisp without any display context.  See
the Lisp callers of the function font-shape-gstring.  One use case is
when we want to display the composition information for a grapheme
cluster to the user, see descr-text.el (used by the "C-u C-x ="
command).  In these cases, the UBA is not invoked, and so we don't
have the direction information.

I could provide the direction information in this case by using the
directionality of the base character of the grapheme cluster, but I
figured out that HarfBuzz already does this as part of its guessing.
Doesn't it?

> I pushed a couple of commits that does this based on my limited
> understanding of Emacs code, please check.

Thanks.  Do you see any difference in the results?  If so, can you
please show the text you used and the results of shaping it with both
versions.  AFAIU, your code should produce exactly the same results,
unless I'm missing something.  (I didn't want to use the
resolved_level attribute because it is ephemeral, and might not
provide the correct value where we are using it.)

Btw, did you test both paragraph directions (controlled by the
bidi-paragraph-direction variable), and also text inside directional
override which changes its natural direction?