[ft] Re: Texts Rasterization Exposures

[David Turner]

Hi Maxim,

On Wed, 11 Jul 2007 11:32:11 -0400, address@hidden said:
> Hi David, 
> 
> Thanks for reading the article and Many thanks for your comprehensive
> response!
> I'll reply quickly.
> 
> 1. I agree that the backward compatibility is a real curse. But 5 years
> ago, when MS introduced ClearType, it was *very* possible to provide the
> possibility of accurate layout making, *along* with the existing API.
> They didn't do that. I'm sure if they did, the situation with scalable
> forms would be much better nowadays. Besides, You can take Adobe Acrobat
> Reader and turn AA off. Everything remains readable while preserving the
> layout. It means it's very possible with all methods, B&W mono,
> grayscale, and RGB sub-pixel.
> 
Acrobat does a lot of subtle things with text, for example they implement
sub-pixel hinting[1] (i.e. they hint sub-pixel positioned linearly stretched
vectors), they also change the inter-word spacing, and can expand/contract
glyph shapes as well to better fit a paragraph line width.

generally speaking, high-quality text layout is a subtle thing that requires
some pretty heavy computations to do well. You can read the explanations of
the TeX layout algorithm for an overview. All of this is rather complex and
slow, and I can understand why this is not part of any standard text rendering
engine on any consumer-grade operating system, OS X included.

Apart from that, I believe that Microsoft is obsessed with backwards legacy
and that's probably the reason they didn't want to touch their system. E.g. note
that it's only in Vista that ClearType became the default font rendering method.

> 2. Thanks for the information about the light hinting! I'll add an update
> to the article. However, this horizontal stretch "cheating" remains
> actual for the monospaced fonts, such as Courier. I use
> face->glyph->linearHoriAdvance, so, it should be accurate and strictly
> proportional to the sub-pixel height I set. However, it snaps the
> "advance" to some value.
> 
This definitely looks like a bug, because the linearHoriAdvance field is 
designed
precisely to return a non-snapped value. Could you give me a concrete example of
the problem, so I can have a look at it ?

> 3. The screenshot you provided looks really good compared to most of
> Linux screenshots. I understand, the method of RGB sub-pixel filtering is
> an "untouchable holy cow", but still, may be it makes sense to add an
> alternative method, such as Steve Gibson describes? 
> 
Ah, the screenshot provided are using a better-than-Gibson filter that is
part of the patches available at http://david.freetype.org/lcd/

so in a certain sense, this alternative method is already available. It simply
hasn't been included in upstream versions of libXft and Cairo by their 
maintainers,
even though some distributions now use my patches in the packages they 
distribute.

Keith Packard has already stated it didn't want such a filter for libXft because
it widens the bitmaps, and introduces complexities he prefers clients not to 
care
about. There is also the fact that the "legacy" filter has more contrast is you 
have
highly-hinted truetype fonts (i.e. if you use the bytecode interpreter)

As for Cairo, I suppose this is mainly to be consistent with the libXft 
implementation
(Qt uses libXft, Gnome uses Cairo, so having two different implementations may 
lead to
surprising results on a typical desktop).

> 4. In my opinion the X11 committee should seriously consider sub-pixel
> capabilities in the protocol. 
> 
the X11 commitee has nothing to do about anti-aliased text on Unix desktops.

All the work comes from the XRender extension, one of Keith Packard's 
brainchilds,
which already supports sub-pixel positioning; that's because, by design, it is 
up
to the client to send the pixmaps it wants to be alpha-blended, as specific
positions.

it may be possible to augment the extension a little to support things like LCD
sub-pixel positioning though. that's an interesting concept, but you can do the
same at the moment with client code (and bigger client-side caches)

Hope this helps,

- David

[1] the FreeType auto-hinter supports this as well, but since I don't think any
    serious library is going to use it before a loooooong time, I didn't expose
    it through a public API.


> Hello Maxim,
> 
> On Sun, 8 Jul 2007 16:20:50 -0400, "Maxim Shemanarev"
> <address@hidden> 
> said:
> > Hi David,
> > 
> > I'm the author of the Anti-Grain Geometry project, and we have a contact 
> > with you about 3 years ago. 
> 
> Yes, I remember pretty well. Good to be in contact again..
> 
> > I finally tried to summarize my experience and observations concerning the
> > situation with text rasterization in Windows and Linux. The article also 
> > contains demo applications to play with my method of RGB sub-pixel text
> > rendering. I admit some statements may  sound questionable, so, I appreciate
> > any comments, criticism, and  discussions. So, here it is:
> > http://antigrain.com/research/font_rasterization/index.html
> > 
> > It would be very interesting to learn about your opinion. The simple method 
> > I propose may be of potential interest in the GNU/Linux community.
> > 
> ok, I've read your page, and here are my remarks, in no relevant order:
> 
> - I think you're a bit harsh on Microsoft, most of the choices they've
> made are 
> due
>   to backwards compatibility. *Many* applications assume integer advance 
> widths
>   and caret position. they wouldn't work correctly with sub-pixel
>   positioning 
> and
>   lack of hinting. 
> 
>   Apple didn't have this problem when they wrote OS X, because they
>   didn't 
> intend
>   to support old applications except in a sort of virtualized way, so
>   they 
> could
>   start from a fresh codebase and a completely redesigned text subsystem.
> 
>   And the same problem will happen on Linux if you want to implement
>   sub-pixel
>   positioning, because most of the GUI toolkits out there cannot deal
>   with it 
> correctly
>   (and due to the way toolkits are highly fragmented, the problem isn't
>   going 
> to go
>   away easily). And some applications will not work correctly anymore if
>   you 
> change
>   the toolkit. Backwards compatibility is a bitch :-)
> 
> - LCD rendering introduces its own sort of challenges. For example, the
> RGB 
> filtering
>   tends to create wider bitmaps. What this means is that coding something
>   like 
> a terminal
>   correctly becomes more challenging due to overlapping glyph boxes which
>   do 
> not happen
>   normally with monospaced fonts. that's exactly why the libXft and Cairo 
> default LCD filters
>   only touch intra-pixel components: Keith Packard wanted to avoid to
>   deal with 
> this problem,
>   or it would have needed changes in xterm and others, and he didn't want
>   to do 
> that.
> 
>   Funnily, I've provided patches for better filtering a long time ago
>   (see the 
> dates at:
>   http://david.freetype.org/lcd for example), and this doesn't seem to
>   create 
> much problems
>   in practice, except for the occasionnal color fringe on the borders of 
> selected text and
>   stuff like that...
> 
>   As a related note, some Windows editors (e.g. Ultra-Edit) even have
>   special 
> options to
>   deal with ClearType (which in XP, still uses hinting and integer
>   advances) 
> because of this.
> 
> - Windows Vista supports sub-pixel positioning just fine (even without 
> ClearType), but
>   this is "reserved" to WPF applications only. Unsurprisingly, because
>   the 
> framework was
>   designed with device-independence from the start. this also explains
>   why you 
> *cannot*
>   disable anti-aliasing in WPF applications by the way.
> 
> - Microsoft Word is known to use font metrics in a weird way. Basically,
> it 
> uses the
>   hinted metrics corresponding to your printer's resolution to compute
>   text 
> layout.
>   I'm not making this up. When it displays text on your page, it places
>   it by 
> scaling
>   the printer metrics to the screen, which introduces all sorts of
>   irregular 
> spacing.
> 
>   of course, the really idiotic thing is that if you change your printer 
> setting, it will
>   reflow the document (and if you don't have any printer, it chooses a
>   default 
> resolution,
>   which I don't remember here) in a different way.
> 
>   that's *very* stupid now, but I guess they did it for Word 1.0, where
>   it was 
> better to
>   produce text that would fit well on a dotted daisy printer, which
>   required 
> very strong
>   hinting to produce good results... Again, I smell backwards
>   compatibility.
> 
>   However, you should *never* use Word as a comparison for text
>   rendering, 
> because this
>   issue makes any of your comments pointless, unless you know exactly
>   what 
> you're speaking
>   of.
> 
> - Something you may not know is that Adobe Acrobat Reader does weird
> things to 
> font
>   sometimes. It likes to slightly expand/contract glyph shapes to make
>   them 
> better fit
>   the total paragraph width (not counting inter-word spacing). this is
>   all 
> pretty advanced
>   stuff, but not always suitable for general application frameworks (it
>   may 
> also explain
>   why text rendering is so slow in this program as well).
> 
>   Again, be careful when you use this program for comparison purposes...
> 
> - With FreeType's auto-hinter, it's possible to "correct" the
> inter-character 
> spacing to
>   better reflect the original glyph shapes. this process is called 
> "auto-kerning" or
>   "device-kerning", and consists of looking at the space between two
>   glyphs 
> before and
>   after hinting. If the difference is too wide or too short, you can add
>   or 
> remove a
>   spacing pixel for better results.
> 
>   I've added the "ftdiff" program to the ft2demos package to demonstrate
>   this, 
> also look at
>   the following pictures:
> 
>      http://david.freetype.org/freetype/why-deltas-matter-1.png
>      http://david.freetype.org/freetype/why-deltas-matter-2.png
>      http://david.freetype.org/freetype/why-deltas-matter-3.png
> 
>   the left column is unhinted and sub-pixel positioned text.
>   the middle column is normal hinted text with auto-kerning enabled
>   the right column is normal hinted text, without auto-kerning
> 
>   auto-kerning is pretty simple to implement, it boils down to using the 
> "rb_delta" and
>   "lb_delta" returned by the auto-hinter.  Look at the end of the
>   following 
> section:
> 
>     
> http://www.freetype.org/freetype2/docs/reference/ft2-base_interface.html#FT_Glyp
> hSlotRec
> 
>   the algorithm is simply:
> 
> =======================================================================
>     FT_Pos  origin_x       = 0;                                      
>     FT_Pos  prev_rsb_delta = 0;                                      
>                                                                    
>                                                                    
>     for all glyphs do                                                
>       <compute kern between current and previous glyph and add it to 
>        `origin_x'>                                                   
>                                                                    
>       <load glyph with `FT_Load_Glyph'>                              
>                                                                    
>       if ( prev_rsb_delta - face->glyph->lsb_delta >= 32 )           
>         origin_x -= 64;                                              
>       else if ( prev_rsb_delta - face->glyph->lsb_delta < -32 )      
>         origin_x += 64;                                              
>                                                                    
>       prev_rsb_delta = face->glyph->rsb_delta;                       
>                                                                    
>       <save glyph image, or render glyph, or ...>                    
>                                                                    
>       origin_x += face->glyph->advance.x;                            
>     endfor                             
> ==========================================================================
> 
>   and since the distortions are always very small, you can also speed it
>   up
>   by removing all the tests with:
> 
>        delta = prev_rsb_delta - face->glyph->lsb_delta;
>        origin_x += (delta + 32) & ~63;
> 
>   (and of course, the rsb_delta and lsb_delta are always 0 for monospaced 
> fonts).
> 
>   And yes, this has been available from FreeType since quite some time
>   now :-)
> 
> - the trick of multiplying the Y resolution by 100, then scaling down the 
> result is not
>   very elegant. You'd better load the glyphs twice (once hinted, the
>   other 
> not), and merge
>   the result (take the X values from the hinted result, the Y values from
>   the 
> unhinted ones)
> 
> - what surprises the most is that you don't seem to be aware of
> FreeType's 
> capabilities. What
>   you're asking for (hinting only in the vertical direction) *already*
>   exists 
> and is named
>   "light hinting", in FreeType parlance. Most Linux distributions already 
> implement something that
>   is very very near what you're asking: go to the Gnome Font Control
>   Panel, 
> select "Advanced",
>   then select "Light hinting" and "Subpixel Rendering".
> 
>   on recent distros, this will use the light hinter + LCD filtering. the
>   only 
> difference to your scheme
>   is that sub-pixel glyph positioning is not implemented. However, this
>   is a 
> problem of the frameworks
>   on top of FreeType, not the font engine itself (you can trivially
>   translate 
> the result of the light
>   hinter yourself to achieve sub-pixel positioning). The result is still
>   quite 
> good in my opinion.
>   For example, see http://david.freetype.org/lcd/lcd-filter-1.png, that's
>   what 
> I have on one of my
>   desktop, not some fancy mockup.
> 
>   All other things you ask from the font engine have been available from
>   a long 
> time (the real linearly
>   scaled advanced width, the linearly scaled kerning values, etc...).
>   Please 
> consult the documentation,
>   I think you'll find it interesting.
> 
> - nonetheless, there is still a lot of work to do on top of the font
> engine to 
> make text really better.
>   If I had the time to start a "next-generation text rendering"
>   initiative for 
> Linux, I would probably
>   list it as follows:
> 
>      - add support for sub-pixel positioning to most text layout toolkits
>      used 
> by a typical Linux
>        desktop. this means modifying Pango, LibXft, Qt, Cairo, at the
>        very 
> least. Probably also some
>        other libraries like poppler. I'm probably forgetting a lot.
>        Things like 
> Sun's JavaVM also
>        perform their own font rendering and do not respect standard
>        setting. 
> this is gross and only
>        leads to inconsistencies that make users cry.
> 
>      - add support for kerning and auto-kerning to the same toolkits
>      (some 
> support hinted kerning,
>        which isn't very useful). auto-kerning is only useful when using 
> hinting, ignore it otherwise.
> 
>      - LCD filtering/rendering is currently implemented in several
>      libraries 
> (libXft and Cairo), and
>        it would be better if it was located in a single library. Since
>        there 
> are patent issues about
>        it (see http://david.freetype.org/cleartype-patents.html), I would 
> suggest placing this code
>        in FreeType itself. Just like the TrueType native interpreter, it
>        would 
> be disabled by default
>        builds of the library, but could be enabled by recompiling a
>        single 
> package.
> 
>        oh, and allow for better filtering than the current default in the 
> upstream libraries.
> 
>      - implement gamma-correct alpha-blending when displaying text.
>            basically needs changes to the XServer implementation, and
>            probably 
> to the XRender protocol
>            to be done 100% correctly. So it's both hard and *very*
>            difficult to 
> get accepted/implemented.
>            Can be approximated by clients otherwise, at the risk of
>            exploding 
> glyph cache footprint and 
>            X11 traffic :-(
> 
>       - the current font preferences dialog in Gnome/KDE/Wathever are
>       also a 
> joke, they expose settings
>         that are too obscure to most users. I would re-design them to
>         present 
> something more understandable
>         to ordinary people and hackers alike.
> 
>   Alas, I lack the time and motivation to do all this work. If you feel
>   like 
> it, feel free to start such
>   an initiative (find a good name first, and please avoid stupid
>   acronyms). I 
> have provided many patches
>   in the past that are mostly ignored by the corresponding library's 
> maintainers (but are surprisingly
>   included by distribution package managers), and I'm quite tired of
>   this.
> 
> Best Regards,
> 
> - David Turner
> - The FreeType Project  (www.freetype.org)
> 
> > McSeem