[ft-devel] [RFC] design of per-site limiter of memory allocation

[mpsuzuki]

Hi all,

During the new year holiday, I was trying to introduce per-site limiter
of memory allocation to memory debugger module in FreeType2. I think
it might be useful for the application programmers to reproduce the
out-of-memory error by specific scenario. If you're interested in,
please give me comment about the current design.

The patch is attached: mps20110107-ft2_limit-alloc-per-site.diff
--------------------------------------------------------------------
* What is this?
Current FreeType2 memory debugger provides 2 interfaces to restrict the
memory allocator; FT2_ALLOC_TOTAL_MAX (historical summation of allocation,
free() is not considered) and FT2_ALLOC_COUNT_MAX (how many times allocation
is requested). Their restriction is for whole of FT2 library, so it is
difficult to make an OOM error occur at specific site.

During the debug for Savannah bug#31923, I wanted to make OOM error in
ftc_sbit_copy_bitmap().

Attached patch introduces 2 interfaces to define the per-site limiter
of memory allocation; FT2_ALLOC_TOTAL_MAX_SITE and FT2_ALLOC_CUR_MAX_SITE.
FT2_ALLOC_TOTAL_MAX_SITE is a per-site version of FT2_ALLOC_TOTAL_MAX.
FT2_ALLOC_CUR_MAX_SITE is a modified version of FT2_TOTAL_MAX_SITE to
consider the effect of free().

* Basic syntax of the environmental variables to control per-site limiter
As FT2_DEBUG is a space-separated list, new interfaces are same.
A limiter for a specific site is described by a pair of site-specifier
and the max buffer size allocated by the site.
        <limiter> ::= <a_site>,<max_buffer_size>

        <site> ::= <pathname>:<line_number>

        <pathname> ::= free string
        <line_number> ::= 32-bit integer

        <max_buffer_size> ::= 32-bit integer

and FT2_ALLOC_{TOTAL,CUR}_MAX_SITE should be set as a space-separated
list of the limiters.

        <limiters> ::= <limiter> | <limiter> <limiters>

For example,
        FT2_ALLOC_CUR_MAX_SITE="/foo/bah/aaa.c:10,1000 /foo/bah/bbb.c:21,10000"

* More easier syntax is required?
The site specifier is embedded by existing memory debugger. You can the
summary report when you set FT2_DEBUG_MEMORY environment, like this:

FreeType Memory Dump: current=0 max=1360517 total=3047988 count=0
 block  block    sizes    sizes    sizes   source
 count   high      sum  highsum      max   location
-------------------------------------------------
     0   6879        0   385224       56 
/home/mpsuzuki/redhat/BUILD/freetype2-current/freetype2/src/base/ftobjs.c:302
     0   6878        0   385168       56 
/home/mpsuzuki/redhat/BUILD/freetype2-current/freetype2/src/cache/ftcsbits.c:57
     0   6878        0   357656       52 
/home/mpsuzuki/redhat/BUILD/freetype2-current/freetype2/src/base/ftglyph.c:290
     0   6878        0   220096       32 
/home/mpsuzuki/redhat/BUILD/freetype2-current/freetype2/src/cache/ftcimage.c:67
     0    430        0   123840      288 
/home/mpsuzuki/redhat/BUILD/freetype2-current/freetype2/src/cache/ftcsbits.c:232
...

Setting FT2_ALLOC_CUR_MAX_SITE with absolute pathname like
"/home/mpsuzuki/redhat/BUILD/freetype2-current/freetype2/src/base/ftobjs.c:302,1000
 ..."
is lengthy. I want to specify by basename like "ftobjs.c:302,1000 ...".
But FreeType2 is a cross-platform library assuming only C89 and we
should not expect the function like basename(). ft2demos has already it,
slightly modified version is written in another patch;
mps20110107-ft2_basename.diff. Here if libgen.h is available,
native basename() is used. Also raw slash in resource fork accessor
is replaced by new macro PLATFORM_DIR_SEPARATOR.

* Future issues
Some people may request a feature to define the limiter with
more fine-tuned scenario, like "per specific stackframe".
For example, if there is a generic function calling allocater
and the function is called by many places, the site specification
by the position where the allocator is invoked is not enough
to set fine-tuned timebomb. I was trying to add a feature to
define a limiter by the range of code, catching all allocation
from the range. The patch was unfinished, if you're interested
in, please refer the patch uploaded on Savannah bug#31923.

Regards,
mpsuzuki

mps20110107-ft2_limit-alloc-per-site.diff
Description: Text Data

mps20110107-ft2_basename.diff
Description: Text Data