[ft-devel] freetype, undefined behaviour, and clang

[Sean McBride]

Hi all,

I recently read this 3 part blog article by the architect of llvm/clang, which 
I highly recommend:

<http://blog.llvm.org/2011/05/what-every-c-programmer-should-know.html>
<http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_14.html>
<http://blog.llvm.org/2011/05/what-every-c-programmer-should-know_21.html>

In short, clang has two compiler flags that catch undefined behaviour. He 
described them thusly:

"Clang has an experimental -fcatch-undefined-behavior mode that inserts runtime 
checks to find violations like shift amounts out of range, some simple array 
out of range errors, etc. This is limited because it slows down the 
application's runtime and it can't help you with random pointer dereferences 
(like Valgrind can), but it can find other important bugs. Clang also fully 
supports the -ftrapv flag (not to be confused with -fwrapv) which causes signed 
integer overflow bugs to trap at runtime (GCC also has this flag, but it is 
completely unreliable/buggy in my experience)."

For fun, I enabled these flags on the VTK project and ran its unit tests.  Some 
tests now crash, in freetype code.  Example:

ftcbasic.c
360           if ( (FT_ULong)(type->flags - FT_INT_MIN) > FT_UINT_MAX )

(gdb) p type->flags
$1 = 4

On my system, INT_MIN is (-2147483647-1) and so this is in fact an overflow.

I might one day have time to submit patches, but I though I'd point out this 
tool incase any of you would like to take a stab at it.

Cheers,

--
____________________________________________________________
Sean McBride, B. Eng                 address@hidden
Rogue Research                        www.rogue-research.com
Mac Software Developer              Montréal, Québec, Canada