Re: [ft-devel] tests

[George Williams]

On Tue, 2005-02-08 at 19:12, Werner LEMBERG wrote:
> > Take a collection of good input (programs that parse, real fonts,
> > whatevers).
> > Randomly choose one of these.
> > Find a random offset in that file, and randomly change a single
> > character.
> > Parse it.
> > If the parser crashes, save the file.
> > Repeat.
> 
> I had something like this in my mind too to test the parser...  Does
> someone volunteer to set up the proper framework?
Hi Werner,
   I have here a rather basic little program which (I think) does what I
described. Depending on how this is compiled it will either test
fontforge or freetype.

The superstructure expects a set of directories containing good fonts,
and a set of extensions of fonts to be tested. It will randomly pick a
font, copy it, introduce and error and then test it.

   The freetype tests are quite basic:
        For each erroneous font it:
                forks off a new tester
                Initializes the library
                Opens each font in the file
                Loads each glyph
                        (optionally reviewing the contours of the glyph)
                        (optionally rasterizing)
                Closes the face
If the tester exits with a signal, or takes longer than an hour then the
superstructure saves the erroneous font and continues.
If the tester exits normally or with an error, then the superstructure
removes the test font and continues.

Arguments are:
        --dir <dir>     Add this directory to the list of directories
                        to search for good fonts
        --ext <ext>     Add this extension to the set of font types
                        tested.
        --all           Test every font in the directory(ies) no matter
                        what its extension (some cid keyed fonts have
                        no extension)
        --result <dir>  This is the directory in which test files are
                        placed.

        --test <file>   Run a single test on a pre-generated testcase.
                        Done in the current process so it can be
                        debugged more easily.

        --error-count <cnt> Introduce this many single-byte errors into
                        the erroneous fonts.
        --error-fraction <frac> Multiply the file size of the font by
                        this fraction, and introduce that many errors
                        into the erroneous font file.

        --rasterize     Do an FT_Render_Glyph as well as loading it
        --nohints       Specify FT_LOAD_NO_HINTING when loading glyphs
        --check-outlines Call FT_Outline_Decompose on each glyph

        --usage         Prints out this list of options.

After playing with this with fontforge it appears
        A single byte error usually makes a pfa/pfb file totally
         unusable (not surprising considering how the encryption
         works)
        A single byte error usually is unnoticed in a ttf file.

Hope this is of some help. 

Feel free to change it however seems appropriate -- especially to change
the license text to match freetype's requirements.

randomtest.tgz
Description: GNU Zip compressed data