[Devel] Bitmap font metric issues in FreeType

[Keith Packard]

I'm trying to avoid ever shipping .bdf/.pcf files from freedesktop.org and
so I'm using Juliusz's fonttosfnt application to convert the existing bitmap
fonts.  I aspire to make .ttf fonts loaded into the X server act exactly like
the same font loaded from a .bdf file..  I've run into some "issues" over
font metrics and labeling within FreeType and the .ttf file.

BDF fonts have essentially four different vertical metrics:

        1)      Nominal point size (POINT_SIZE)
        2)      Nominal pixel size (PIXEL_SIZE)
        3)      Font ascent/descent (FONT_ASCENT/FONT_DESCENT)
        4)      Maximum glyph ascent/descent (max over all glyphs)

  The max ascent/descent values are "uninteresting" in terms of spacing 
  the font or naming the font size, so we'll ignore them from here on out.

TTF has n different metrics:

        1)      Unscaled ascent/descent/gap in EMs ('hhea' Ascender/Descender)
        2)      Strike ascent/descent in pixels ('EBLC' line metrics ascender/
                descender)
        3)      Strike ppemX/ppemY in pixels ('EBLC' ppemX/ppemY)

FreeType has 3 different metrics:

        1)      bitmap "sizes"     (face->available_sizes[n])
        2)      Current size       (face->size.metrics)
        3)      Current v metrics  (face->ascender, face->descender, 
face->height)

  Here's what's inside the first two:

    face->available_sizes (FT_Bitmap_Size):

        height          baseline-to-baseline distance in pixels         (<< 0)
        width           average width in pixels                         (<< 0)
        size            nominal point size                              (<< 6)
        x_ppem          horizontal pixels-per-EM                        (<< 6)
        y_ppem          vertical pixels-per-EM                          (<< 6)

    face->size->metrics (FT_Size_Metrics):

        x_ppem          horizontal pixels-per-EM                        (<< 0)
        y_ppem          vertical pixels-per-EM                          (<< 0)
        ascender        ascent in pixels                                (<< 6)
        descender       descent in pixels                               (<< 6)
        max_advance     maximum 'width' in pixels                       (<< 6)

  And, to mess things up further, the current v metrics are not defined 
  when loading a bitmap font (TTF sets them, BDF does not).

Ok, now let's see what FreeType does with our BDF font.  It must set the 
available_sizes values, and size->metrics values:

    BDF face->available_sizes (FT_Bitmap_Size)

                height = PIXEL_SIZE
                width = AVERAGE_WIDTH
                size = POINT_SIZE * constant
                x_ppem = RESOLUTION_X * POINT_SIZE / 72
                y_ppem = RESOLUTION_Y * POINT_SIZE / 72

        height is wrong (should be FONT_ASCENT + FONT_DESCENT)
        width is correct
        size is correct
        y_ppem is not quite right (should be PIXEL_SIZE)
                Note that this will be close to how it is computed, modulo 
                rounding errors
        x_ppem could be computed differently 
                (PIXEL_SIZE * RESOLUTION_X / RESOLUTION_Y)?

   BDF face->size->metrics (FT_Size_Metrics) (valid afer FT_Set_Char_Size)

                x_ppem = char_width passed to FT_Set_Char_Size 
                                        (adjusted for h_res)
                y_ppem = char_height passed to FT_Set_Char_Size 
                                        (adjusted for v_res)
                ascender = maxbounds.ascent
                descender = -maxbounds.descent
                height = maxbounds.height
                max_advance = maxbounds.width

        That FT_Set_Char_Size actually wants the EM-box dimensions is
                somewhat confusing.  Fontconfig stores the height values 
                instead of the EM-box values, so I'll have to fix that.

        x_ppem/y_ppem are right (by definition, it's global FreeType code 
                which sets them)
        ascender is wrong (should be font_ascent)
        descender is wrong (should be font_descent)
        height is wrong (should be font_ascent + font_descent)
        max_advance is correct

        One other issue here is that the BDF loader compares the
        specified height (stored in y_ppem) against the 
        available_sizes.height value.

Now, let's look at what FreeType will do with a bitmap TTF font.  Again, 
it must set the available_sizes and size->metrics values:

    TTF face->available_sizes (FT_Bitmap_Size)

           hhea.Ascender = unscaled ascender from 'hhea' table
           hhea_height = hhea.Ascender - hhea.Descender + hhea.Line_Gap

                height = hhea_height * (Strike ppemY)
                width = (os2 average char width) * (Strike ppemX)
                size = (Strike ppemY)
                x_ppem = (Strike ppemX)
                y_ppem = (Strike ppemY)

        height is wrong (should be Strike ascender + Strike Descender + ?)
        width is right
        size is kinda wrong - it's assuming 72 dpi (hardly matters, I guess)
        x_ppem/y_ppem are correct
        
        The problem with height is that using the global metrics is
        incorrect as that will introduce significant rounding errors.
        However, the strike doesn't include any Line_Gap value, so
        this computation does make some sense.  I suggest using the
        global Line_Gap value and the strike ascender/descender values.
        For our BDF fonts, Line_Gap will always be zero, so this will work
        fine.

    TTF face->size->metrics (FT_Size_Metrics) (valid after FT_Set_Char_Size)

                x_ppem/y_ppem = values passed to FT_Set_Char_Size 
                                                (adjusted for res)
                ascender = (Strike ascender)
                descender = (Strike descender)
                height = (Strike ascender - Strike descender)
                max_advance = (something hard, marked with 'XXX')

        x_ppem/y_ppem are (again, set by FreeType base) correct by definition
        ascender is correct
        descender is correct
        height is almost correct; it ignores Line_Gap, which (it appears)
                bitmap fonts don't get to have
        max_advance may well be wrong, it it's probably good enough

Ok, so now that I know what FreeType is doing, the first question is how 
to fix (or work around) the problems.  Let's start with the assumption 
that the BDF loader is the most broken; it's newest, and least well tested.
Plus, we can change it in arbitrary ways and not hurt anyone who has 
correct TTF fonts.

I suggest that the two metrics (available_sizes and face->size->metrics) 
should be reconciled so that one can use available_sizes to compute what 
values will appear in face->size->metrics, and reliably select character 
sizes based on those values.  

The values passed to FT_Set_Char_Size/FT_Set_Pixel_Sizes must be the EM box
values, that's (nominally) what the 'point size'/'pixel size' of a font
names.  So, the first thing to fix is to change the BDF driver to match 
the x_ppem/y_ppem values instead of width/height.  Simple enough.  The 
remaining BDF computations look easy enough to fix; that's not urgently 
needed in a FreeType release as I can create the TTF files with a fixed 
version and not run afowl of the existing BDF reader on other peoples 
boxes.

Now, the TTF loader is scrambling the height values in the available_sizes 
array, but I mostly don't care; fontconfig uses available_sizes only for 
recording the list of pixel sizes, which is nicely stored (correctly) in 
ppem_y.

I've attached a patch which makes the BDF driver behave as described 
above, I haven't taken a stab at fixing the TTF driver yet.

-keith

bdf-driver.diff
Description: bdf-driver.diff