bug#33729: 27.0.50; Partial glyphs not rendered for Gujarati with Harfbuzz enabled (renders fine using m17n)

[Khaled Hosny]

On Mon, Dec 24, 2018 at 06:10:49PM +0200, Eli Zaretskii wrote:
> > Date: Mon, 24 Dec 2018 04:08:47 +0200
> > From: Khaled Hosny <dr.khaled.hosny@gmail.com>
> > Cc: rgm@gnu.org, far.nasiri.m@gmail.com, behdad@behdad.org,
> >     33729@debbugs.gnu.org, kaushal.modi@gmail.com
> > 
> > I think we are almost good now. There is only one serious FIXME left:
> > 
> >   /* FIXME: guess_segment_properties is BAD BAD BAD.
> >    * we need to get these properties with the LGSTRING. */
> > #if 1
> >   hb_buffer_guess_segment_properties (hb_buffer);
> > #else
> >   hb_buffer_set_direction (hb_buffer, XXX);
> >   hb_buffer_set_script (hb_buffer, XXX);
> >   hb_buffer_set_language (hb_buffer, XXX);
> > #endif
> > 
> > We need to know, for a given lgstring we are shaping:
> > * Its direction (from applying bidi algorithm). Each lgstring we are
> >   shaping must be of a single direction.
> 
> Communicating this to ftfont_shape_by_hb will need changes in a couple
> of interfaces (the existing shaping engines didn't need this
> information).  I will work on this soon.

Great.

> > * Its script, possibly after applying something like:
> >   http://unicode.org/reports/tr24/#Common
> 
> Per previous discussions, we decided to use the Harfbuzz built-in
> methods for determining the script, since Emacs doesn't have this
> information, and adding it will just do the same as Harfbuzz does,
> i.e. find the first character whose script is not Common etc., using
> the UCD database.  I think it was you who suggested to use the
> Harfbuzz built-ins in this case.

The built-in HarfBuzz code is for getting the script for a given
character, but resolving characters with Common script is left to the
client. Suppose you have this string (upper case for RTL) ABC 123 DEF,
what HarfBuzz sees during shaping is three separate chunks of text ABC,
123, DEF. The 123 part is all Common script characters and thus
hb_buffer_guess_segment_properties won’t be able to guess anything (and
based on the font and the script, this can cause rendering differences).
Emacs will have to resolve the script of Common characters before
applying bidi algorithm and pass that down to HarfBuzz.

> > * Its language, is Emacs allows setting text language (my understand is
> >   that it doesn’t). Some languages really need this for applying
> >   language-specfic features (Urdu digits, Serbian alternate glyphs, etc.).
> 
> We don't currently have a language property for chunks of text, we
> only have the current global language setting determined from the
> locale (and there's a command to change that for Emacs, should the
> user want it).  This is not really appropriate for multilingual
> buffers, but we will have to use that for now, and hope that in the
> future, infrastructure will be added to allow more flexible
> determination of the language of each run of text.  (I see that
> Harfbuzz already looks a the locale for its default language, but
> since Emacs allows user control of this, however unlikely, I think
> it's best to use the value Emacs uses.)  I will work on this as well.

Yes, better pass that from Emacs to HarfBuzz.

Regards,
Khaled