hashing macros in ftcache.h (Re: [ft] freetype2 and Win64)

[mpsuzuki]

Dear David and Werner,

The hashing macros in ftcache.h should be kept in
future release? I propose to enclose them by the
conditional FT_CONFIG_OPTION_OLD_INTERNALS.
Please give me your comments on following observation
about the macros.

--

The public header file "include/freetype/ftcache.h"
has a few cpp macros to calculate hash values for
the FT2 cache subsystem:

#define FT_POINTER_TO_ULONG( p )  ( (FT_ULong)(FT_Pointer)(p) )

#define FTC_FACE_ID_HASH( i )                                \
          ((FT_UInt32)(( FT_POINTER_TO_ULONG( i ) >> 3 ) ^   \
                       ( FT_POINTER_TO_ULONG( i ) << 7 ) ) )

#define FTC_IMAGE_TYPE_HASH( d )                          \
          (FT_UFast)( FTC_FACE_ID_HASH( (d)->face_id )  ^ \
                      ( (d)->width << 8 ) ^ (d)->height ^ \
                      ( (d)->flags << 4 )               )

#define FTC_FONT_HASH( f )                              \
          (FT_UInt32)( FTC_FACE_ID_HASH((f)->face_id) ^ \
                       ((f)->pix_width << 8)          ^ \
                       ((f)->pix_height)              )

FT_POINTER_TO_ULONG() should be dangerous for the
calculation related with the memory object identify
(like hash key for cached object). On LLP64 system,
like Win64, the cast from the pointer to unsigned
long may cause the overflow and the conflict of
hash key for different objects.

In the latest FT2, there is no public functions
taking the hash values calculated by FT2 client.
Either there is no structure that FT2 client can
get the hashed value calculated in FT2 cache sub-
system. Furthermore, FTC_IMAGE_TYPE_HASH() and
FTC_FONT_HASH() macros are not used in anywhere.

The results of Google codesearch of FTC_FACE_ID_HASH(),
FTC_IMAGE_TYPE_HASH() and FTC_FONT_HASH() are
only the duplicated sources of FreeType2.

        http://www.google.com/codesearch?as_q=FTC_FACE_ID_HASH
        http://www.google.com/codesearch?as_q=FTC_IMAGE_TYPE_HASH
        http://www.google.com/codesearch?as_q=FTC_FONT_HASH

I could not find existing FT2 client implementations
using these macros.

So I wonder why these macros exist in the public
header files. I investigated how these macros were
introduced, I attached the memo. I think these macros
were introduced when FT2 header files had ever
exposed many internal data structures. I guess these
functions were exposed unintentionally, and FT2
cache subsystem had never expected FT2 clients to
calculate the hash values in the client-side.

There had ever been some public functions (FT_EXPORT()
-ed) that takes the hash values from FT2 client (in
the other words, FT2 client can calculate hash values
by itself and pass it to FT2), but all of them are
removed until the version 2.2.

In addition, the change of hashing algorithm was not
considered to break the binary compatibility.

* During version 2.1.3 and 2.1.4, FTC_FACE_ID_HASH()
was changed incompatibly, but docs/CHANGES didn't
mention about it.

* During version 2.1.7 and 2.1.8, FTC_FONT_HASH()
used in FTC_IMAGE_TYPE_HASH() was replaced by
FTC_FACE_ID_HASH() (so, FTC_FONT_HASH() was not
used anymore in later revisions), doc/CHANGES didn't
mention about it.

Considering the history and current little-used
status by FT2 clients, I think the macros are not
required in public header files, so I propose to
enclose them by FT_CONFIG_OPTION_OLD_INTERNALS.

Regards,
mpsuzuki

----------------------------------------------------------------------------
* 2001-10-26 cd605b63022ae135008ce06f0b266cee7b7985e0

Big change of cache system between 2.0.5 and 2.0.6.
3 hash functions (written by cpp macro) were introduced
to public header files:

include/freetype/ftcache.h:
        FTC_FACE_ID_HASH()      defined
        FTC_FONT_HASH()         defined, using FTC_FACE_ID_HASH()
include/freetype/cache/ftcimage.h:
        FTC_IMAGE_DESC_HASH()   defined, using FTC_FONT_HASH()

Other 3 hash functions are introduced to C sources:

src/cache/ftcchunk.c:
        FTC_CSET_HASH()         defined/used to extract a hash from
                                FTC_ChunkSet object and calculate
                                another hash.
src/cache/ftcglyph.c:
        FTC_GSET_HASH()         defined/used to extract a hash from
                                FTC_GlyphNode object and calculate
                                another hash.
src/cache/ftcimage.c
        FTC_IMAGE_DESC_HASH()   used to calculate the hash values
                                to be stored in FTC_ChunkSet and
                                FTC_GlyphNode objects.

FTC_{Chunk|Glyph}Query stuctures include FTC_{Chunk|Glyph}Rec, they
have the embers to store the hash values, so it is possible for FT2
client to pass the hash values claculated in client-side to FT2 library,
via following functions:
        ftc_chunk_cache_lookup( FTC_ChunkCache, FTC_ChunkQuery, FTC_ChunkNode );
        ftc_glyph_cache_lookup( FTC_GlyphCache, FTC_GlyphQuery, FTC_GlyphNode );
----------------------------------------------------------------------------
* 2001-12-05 14183ea0f8b9c021400d748d7311ec2a907e9f8d

Big change of cache subsystem between 2.0.5 and 2.0.6.
New hash function FTC_GLYPH_FAMILY_HASH() is introduced
to the public header.
ftcchunk.{c,h} are (to be) replaced by ftccache.{c,h},
but slipped to be committed, next changeset do that.

include/freetype/cache/ftcglyph.h:
        FTC_GLYPH_FAMILY_HASH() defined
src/cache/ftcglyph.c:
        FTC_GSET_HASH()         removed, replaced by FTC_GLYPH_FAMILY_HASH()
        FTC_GLYPH_FAMILY_HASH() used via FTC_GLYPH_FAMILY_FOUND()
src/cache/ftcimage.c:
        FTC_GLYPH_FAMILY_HASH() used via FTC_GLYPH_FAMILY_FOUND()

In this revision, the functions which may receive the
hash values calculated by FTC_IMAGE_DESC_HASH():
        ftc_chunk_cache_lookup( FTC_ChunkCache, FTC_ChunkQuery, FTC_ChunkNode );
        ftc_glyph_cache_lookup( FTC_GlyphCache, FTC_GlyphQuery, FTC_GlyphNode );
are removed (FTC_ChunkQuery type itself is removed), and replaced by
following functions (see next commit):
        ftc_cache_lookup( FTC_Cache, FTC_Query, FTC_Node* );
        ftc_family_init( FTC_Family, FTC_Query, FTC_Cache );
        ftc_glyph_family_init( FTC_GlyphFamily, FT_UInt32 hash, FT_UInt,
                                FT_UInt, FTC_GlyphQuery, FTC_Cache );
FTC_Query has the member to store the hash value, so still
it is possible for FT2 client to pass the hash value
calculated by FTC_GLYPH_FAMILY_HASH() and FTC_IMAGE_DESC_HASH().

----------------------------------------------------------------------------
* 2001-12-06 a09764555c60459fe35d77e85ba13af62a80b0b0

ftcchunk.{c,h} are replaced by ftccache.{c,h}.

src/cache/ftcchunk.c:
        FTC_CSET_HASH()         removed.

ftc_cache_lookup()
        the hash in passed FTC_Query is initialized,
        no need to calculate/set in FT2 client side.
ftc_family_init()
        the hash in passed FTC_Query is not used at all,
        only query->family is modified.
ftc_glyph_family_init()
        invoke ftc_family_init(), if successfully returned,
        FTC_GlyphFamily->hash is set to passed hash.

So, it is possible for FT2 client to pass the hash value
calculated by FTC_GLYPH_FAMILY_HASH() to ftc_glyph_family_init(). 
----------------------------------------------------------------------------
* 2001-12-07 145f94cb261afc5bdbd924b130cd5d856c42e4b9

The extention of cache subsystem to support charmap
between 2.0.5 and 2.0.6.

src/cache/ftccmap.c:
        FTC_CMAP_HASH()         defined/used

A cmap subtable in TrueType can be identified by the platform ID
and the encoding ID, so no need to introduce a hash function in
public interfaces.
----------------------------------------------------------------------------
* 2002-09-08 00d9f40cf7ec068286792eb352971cd7ed53fc73

The replacement of FTC_ImageDescRec by FTC_ImageTypeRec
between 2.1.2 and 2.1.3. The change of structure is small,
the description of cached image is changed from the flags
constants for cache subsystem ftc_image_{format,flag}_XXX
to public FT_LOAD_XXX flags.

include/freetype/cache/ftcimage.h
        FTC_IMAGE_DESC_HASH()   removed, replaced by:
        FTC_IMAGE_TYPE_HASH()   substitutes FTC_IMAGE_DESC_HASH()

src/cache/ftcimage.c
        FTC_IMAGE_TYPE_HASH()   replaces FTC_IMAGE_DESC_HASH()

In the public functions, FTC_ImageCache_Lookup() and
FTC_SBitCache_Lookup() are affected, but no serious change.
----------------------------------------------------------------------------
* 2002-10-25 2d62446da91575ad3315b9b4f304f0429ca0b63d

The change of hash algorithm in FTC_FACE_ID_HASH()
between 2.1.3 and 2.1.4. The original algorithm
calculating a hash from FaceID had ever been a simple
cast from FT_Pointer to FT_UInt32. The changed algorithm
adds some bitshifting calculations.

include/freetype/ftcache.h:
        FTC_FACE_ID_HASH()      changed internally
----------------------------------------------------------------------------
* 2003-12-19 57ecae22a7b837d486bb03eda944736780ce0868

Big change of cache subsystem between 2.1.7 to 2.1.8.

include/freetype/ftcache.h:
        FTC_IMAGE_TYPE_HASH()   moved from ftcimage.h,
                                replace FTC_FONT_HASH() by FTC_FACE_ID_HASH().
include/freetype/cache/ftcmanag.h
        FTC_SCALER_HASH()       defined, using FTC_FACE_ID_HASH()
include/freetype/cache/ftcglyph.h:
        FTC_GLYPH_FAMILY_HASH() removed.
        ftc_glyph_family_init() removed, replaced by
        FTC_GCache_Lookup()     substitutes ftc_glyph_family_init(),
                                hash is passed by value.

src/cache/ftccache.c:
        ftc_cache_lookup()      removed, replaced by
        FTC_Cache_Lookup()      substitutes ftc_cache_lookup(),
                                hash is passed by value.
        FTC_Cache_NewNode()     hash is passed by value.
src/cache/ftcbasic.c:
        FTC_BASIC_ATTR_HASH()   defined/used, using FTC_SCALER_HASH()
src/cache/ftccmap.c:
        FTC_CMAP_HASH()         changed to reflect the index of cmap subtable.
        FTC_CMAP_QUERY_HASH()   this is not hash function,
                                interface to FTC_CMAP_HASH()
        FTC_CMAP_NODE_HASH()    this is not hash function,
                                interface to FTC_CMAP_HASH()
src/cache/ftcimage.c:
        FTC_IMAGE_TYPE_HASH()   not used anymore.
src/cache/ftcsbits.c:
        FTC_IMAGE_TYPE_HASH()   not used anymore.


FTC_GCache_Lookup() is a thin wrapper of FTC_Cache_Lookup().
FTC_Cache_Lookup() uses the hash (passed by value) as the
index of cache bucket. So it is possible for FT2 clients
to calculate the hash value by themselves and pass it to
FTC_Cache_Lookup().
----------------------------------------------------------------------------
* 2003-12-23 023612221e75e7240316d67a01d80cad069023f6

Cpp macro replacements for FTC_Cache_Lookup() and
FTC_GCache_Lookup() are introduced between 2.1.10 and 2.2.

include/freetype/cache/ftccache.h
        FTC_CACHE_LOOKUP_CMP()  defined
        FTC_GCACHE_LOOKUP_CMP() defined
src/cache/ftccmap.c
        FTC_CMapCache_Lookup()  use FTC_CACHE_LOOKUP_CMP()
src/cache/ftcbasic.c
        FTC_ImageCache_Lookup() use FTC_GCACHE_LOOKUP_CMP()
        FTC_SBitCache_Lookup()  use FTC_GCACHE_LOOKUP_CMP()

None of the functions affected have the interface for
FT2 client to pass the hash values calculated in the client-side.
----------------------------------------------------------------------------
* 2006-03-20 256de4b18f556b963597f171a217c0c1ed045993

Move non-public header files from include/freetype/cache
to src/cache between 2.1.10 and 2.2.

include/freetype/cache/ftccache.h
        FTC_Cache_Lookup()      hidden
        FTC_Cache_NewNode()     hidden
include/freetype/cache/ftcglyph.h
        FTC_GCache_Lookup()     hidden
include/freetype/cache/ftcmanag.h
        FTC_SCALER_HASH()       hidden, moved from header tree to source tree.
----------------------------------------------------------------------------
* 2006-03-22 6c5b617c59a845e198971482b121d56157a16c7a

Cpp macro replacements are enclosed by FTC_INLINE,
between 2.1.10 and 2.2
----------------------------------------------------------------------------