[ft-devel] state of the diagnostics patch b68+6.5, last two error codes, and undocumented engine behaviors

[Hin-Tak Leung]

Hi,

I am about to post another round of libre font test results based on b66 (the 
one that Cosimo asked for, with the corrected maxStackElements check). It is 
probably a good time, coming up to almost exactly a year since FontVal 
open-opened...

The old proprietary FontVal 1.0 backend _documents_ 72 error codes.  I found 
two of them not used AFAIK:
E6017 _rast_E_FDEF_SPACE_NOT_DEFINED
E6041 _rast_E_REFPOINT_USED_BUT_NOT_SET

and one implemented entirely wrongly:
E6019 _rast_E_FUCOORDINATE_OUT_OF_RANGE

The MS code was checking FUnit's bounds as a F26.6, so it was wrongly checking 
for [-1048576, 1048512] instead of  [-16384, 16383] .

I'd be interested to know if E6017/E6041 was ever used, but similar errors are 
covered by various FDEF bound checks and points bound checks so probably not 
important.

The description of 
E6017 ""Function definition space not defined"/"The called function was not 
properly defined. Make sure function was defined in either the font program or 
the pre-program."
is almost identical to
E6020 (which I already use) "Function not defined"/"The called function was not 
properly defined. Make sure function was defined in either the font program or 
the pre-program."

So this appears to be a small bug in FontVal 1.0 (of having defined two nearly 
identical error codes, and not using one of them at all).

There is no detailed description of E6041 in the help file. I'd probably add a 
"HTL: Not used AFAIK" note there one day.

Anyway. As in the title, I am on to b68+6.5, which uses 68 of those 72, plus 11 
new error codes specific to the new backend, 6 on the C side and 5 on the C# 
side. b66 was really a "back-port", the last backend-only update, without 
requiring changes to the C# side of things.

The two last error codes ( 72 - 2 - 68 = 2 ...)  I don't implement are:
E6036 _rast_E_MATH_OVERFLOW
E6040 _rast_E_PREPROGAM_ZONE_NOT_TWI

FreeType does not seem to check 16-bit overflows on ADD/SUB/MUL/DIV , and seems 
to either clamp to max/min, or just rely on downstream to catch it. In reality, 
math overflows, I think, are likely to cause points to fly off to totally wrong 
places, and so are visually noticeable. Such errors are also extremely unlikely 
to be seen in shipping or soon-to-ship fonts. I am unlikely to spend time 
adding that, but if somebody else adds the check to FreeType, I'll hook it up...

E6040 is an interesting one - it is about using real (non-twilight) zones in 
pre-programs. I don't think FreeType does anything special in the circumstances 
- you try to draw real points in prep/cvt/fpgm, doing so trashes every glyph, 
the result is visually noticeable, and you get what you ask for. A check is 
easy to add (unlike the math overflow above), but I don't think it is a problem 
likely to happen by chance and need to catch, as the problem is visually 
noticeable. I hope we are not trying to create (and catch) the most exotically 
broken fonts :-).

Werner: what does FreeType do exactly, in this case?

The reverse, use and mis-use of twilight zones, and reliance on undocumented 
behaviors in twilight zone, is the bulk of the additions beyond b54. Libre 
fonts don't seem to use twilight zone much, and also much of where "XXX 
undocumented" within FreeType's code paths happen.

Here is an overview of everything so far:

Patch 1, the basic c#-to-c hook, plus b06 was as in
http://lists.nongnu.org/archive/html/freetype-devel/2016-07/msg00016.html

Patch 2, b06->b41, adds a whole lot of diagnostics where one of FreeType's 
internal error recovery paths corresponds obviously to one of FontVal 1.0's. It 
is probably the build corresponds closest to how FontVal 1.0 behaves.

Patch 3 plus a few small patches becomes b54, and FontVal 2.0. I assigned 
almost all of FreeType's internal error recovery to one of FontVal error codes. 
Some assignments are generic (value out of range) and some not totally 
appropriate, but all of Freetype's "not happy with this font" is exposed; so 
that's why it was released as final; although there are quite a few errors b54 
catches where the old backend does not, and vice versa.

b54 was not really distinguishable from FontVal 1.0, as far as testing against 
libre font shows. (some differences are seen with proprietary fonts).

The three patches were about 15k each.

Some of the changes beyond b54 are fairly big. Tracking twilight zone usage, 
nested conditional's level, store usage, agreement with maxp parameters etc, 
and about uses and mis-uses of undocumented engine behavior, like how some of 
the truetype instructions work and not work in twilight zones. These are also 
unlikely to benefit libre fonts at all.

The "back-port" b55->b66 diff is 14k. b55 differs from b54 only by re-assigning 
one of the error paths from a more generic error codes to a specific code... 
there is no real difference as far as end-user usage goes, between b54 and b55.