[ft-devel] plan to support sfnt-wrapped CID-keyed font

[mpsuzuki]

Hi,

Recently, sfnt-wrapped CID-keyed font support is discussed in
fontforge-users list (yet sf.net has not updated archive -
please find posts titled "Mac CID Fonts"). It made me interested
in the support by FreeType2, and today I tried to add the feature.
Yet it's not finished (in fact, wrong buffer handling makes
ftdump crashed), but I want to ask a few questions to implement
it properly. In following, I attach my patch for experiment and
describe what the modification does.

As Adobe TechNote #5180 tells, sfnt-wrapped CID-keyed font is not
distributed in naked TTF file format. The file format is used
as a structure of "sfnt" resource of suitcase font format (the
legacy font format of Macintosh: its data fork is empty, and
resource fork includes FOND, NFNT, vers and sfnt resources).
Current FreeType2 can access sfnt resource of suitcase font format.
So, the required part is a bridge from sfnt resource parser to
(naked) CID-keyed font parser (*).

###################################################################

The 1st gap is in the function ttobjs.c::tt_face_init().
It checks the 32bit tag at the beginning of sfnt resource,
and expects the tag is 0x00010000, or 0x00020000, or "true".
But, in sfnt-wrapped CID-keyed font, the tag is "typ1"
(See Adobe TN#5180, Section 2). So, I added a precheck of
"typ1" signature.

Also, I introduced a boolean "is_sfnt_ps" to avoid the
installation of bytecode interpreter for sfnt-wrapped
CID-keyed fonts. Maybe it is NOT essential.

Index: src/truetype/ttobjs.c
===================================================================
RCS file: /sources/freetype/freetype2/src/truetype/ttobjs.c,v
retrieving revision 1.120
diff -u -r1.120 ttobjs.c
--- src/truetype/ttobjs.c       17 May 2008 10:01:45 -0000      1.120
+++ src/truetype/ttobjs.c       29 Aug 2008 09:51:28 -0000
@@ -178,6 +178,7 @@
     FT_Library    library;
     SFNT_Service  sfnt;
     TT_Face       face = (TT_Face)ttface;
+    FT_Bool       is_sfnt_ps = FALSE;
 
 
     library = face->root.driver->root.library;
@@ -197,6 +198,9 @@
     /* We must also be able to accept Mac/GX fonts, as well as OT ones. */
     /* The 0x00020000 tag is completely undocumented; some fonts from   */
     /* Arphic made for Chinese Windows 3.1 have this.                   */
+    if ( face->format_tag == FT_MAKE_TAG( 't', 'y', 'p', '1' ) )
+      is_sfnt_ps = TRUE;
+    else
     if ( face->format_tag != 0x00010000L &&    /* MS fonts  */
          face->format_tag != 0x00020000L &&    /* CJK fonts for Win 3.1 */
          face->format_tag != TTAG_true   )     /* Mac fonts */
@@ -206,6 +210,7 @@
     }
 
 #ifdef TT_USE_BYTECODE_INTERPRETER
+    if ( !is_sfnt_ps )
     face->root.face_flags |= FT_FACE_FLAG_HINTER;
 #endif
 

The 2nd gap is in ttload.c::check_table_dir(), a function to
assue that essential tables are included. It searches head
(standard TrueType), bhed (MacOS sfnt-bitmap), sing/meta
(SING glyphlet font) tables. I'm not sure what sfnt tables
are essential in sfnt-wrapped CID-keyed fonts. Adobe TN#5180
does not tell if "name" "post" etc are required /or not.
In fact, a few samples (e.g. Adobe ValuePack, manufactured
by Adobe) in my hands does not have "head" nor "bhed".

# Adobe TN#5180 tells "BBOX", "CID ", "HFMX", "VFMX" are
# required, but I don't want to check it because they
# (except of CID) are not parsed by FreeType2 at all.

Thus, I added 2 booleans to check if table directory has
"CID " or "typ1" table. If one of them is found,
check_table_dir() permits the font missing "head"/"bhed"
/"SING"/"META". 

Index: src/sfnt/ttload.c
===================================================================
RCS file: /sources/freetype/freetype2/src/sfnt/ttload.c,v
retrieving revision 1.132
diff -u -r1.132 ttload.c
--- src/sfnt/ttload.c   4 Aug 2008 05:45:41 -0000       1.132
+++ src/sfnt/ttload.c   29 Aug 2008 09:51:28 -0000
@@ -149,6 +149,7 @@
     FT_UInt   nn, valid_entries = 0;
     FT_UInt   has_head = 0, has_sing = 0, has_meta = 0;
     FT_ULong  offset = sfnt->offset + 12;
+    FT_Bool   has_cid = FALSE, has_typ1 = FALSE;
 
     static const FT_Frame_Field  table_dir_entry_fields[] =
     {
@@ -236,6 +237,10 @@
         has_sing = 1;
       else if ( table.Tag == TTAG_META )
         has_meta = 1;
+      else if ( table.Tag == FT_MAKE_TAG( 'T', 'Y', 'P', '1' ) )
+        has_typ1 = TRUE;
+      else if ( table.Tag == FT_MAKE_TAG( 'C', 'I', 'D', ' ' ) )
+        has_cid = TRUE;
     }
 
     sfnt->num_tables = valid_entries;
@@ -248,7 +253,7 @@
     }
 
     /* if `sing' and `meta' tables are present, there is no `head' table */
-    if ( has_head || ( has_sing && has_meta ) )
+    if ( has_head || ( has_sing && has_meta ) || ( has_typ1 || has_cid ) )
     {
       error = SFNT_Err_Ok;
       goto Exit;


The 3rd gap is in sfobjs.c::sfnt_open_font(). Although
ttobjs.c::tt_face_init() has already checked the tag,
this function checks it again. So, again I added a precheck
to permit "typ1"-tagged resource. As the 1st gap, I added
a boolean "is_sfnt_ps", but it is not essential at present.


Index: src/sfnt/sfobjs.c
===================================================================
RCS file: /sources/freetype/freetype2/src/sfnt/sfobjs.c,v
retrieving revision 1.133
diff -u -r1.133 sfobjs.c
--- src/sfnt/sfobjs.c   29 Aug 2008 06:09:03 -0000      1.133
+++ src/sfnt/sfobjs.c   29 Aug 2008 09:51:28 -0000
@@ -336,6 +336,7 @@
   /* synthesized into a TTC with one offset table.              */
   static FT_Error
   sfnt_open_font( FT_Stream  stream,
+                  FT_Bool*   is_sfnt_ps,   
                   TT_Face    face )
   {
     FT_Memory  memory = stream->memory;
@@ -357,12 +358,16 @@
     face->ttc_header.tag     = 0;
     face->ttc_header.version = 0;
     face->ttc_header.count   = 0;
+    *is_sfnt_ps = FALSE;
 
     offset = FT_STREAM_POS();
 
     if ( FT_READ_ULONG( tag ) )
       return error;
 
+    if ( tag == FT_MAKE_TAG( 't', 'y', 'p', '1' ) )
+      *is_sfnt_ps = TRUE;
+    else
     if ( tag != 0x00010000UL                      &&
          tag != TTAG_ttcf                         &&
          tag != TTAG_OTTO                         &&
@@ -421,6 +426,7 @@
     FT_Error        error;
     FT_Library      library = face->root.driver->root.library;
     SFNT_Service    sfnt;
+    FT_Bool         is_sfnt_ps;
 
 
     /* for now, parameters are unused */
@@ -441,7 +447,7 @@
 
     FT_FACE_FIND_GLOBAL_SERVICE( face, face->psnames, POSTSCRIPT_CMAPS );
 
-    error = sfnt_open_font( stream, face );
+    error = sfnt_open_font( stream, &is_sfnt_ps, face );
     if ( error )
       return error;




The 4th gap is in sfobjs.c::sfnt_load_face(). It searches "glyf"
or "CFF " tables, the outline glyph tables of standard OpenType
font. If both tables are not found, the function proceeds to
load bitmap tables. Thus, I inserted a fallback to search "CID "
or "TYP1" tables.

"CID " table consists from 2 parts: binary header and standard
CID-keyed font data. FreeType2 CID-keyed font parser can parse
latter.

If "CID " or "TYP1" is found, the stream is seeked to the table
and binary header is parsed (sorry, at present, sfnt-wrapped
Type1 is not supported because I don't have the documents nor
sample fonts). After the binary header is fed, the rest content
of the table is duplicated to the buffer. The buffer is passed
to open_face_from_buffer(), as MacOS LaserWriter PS fonts.
open_face_from_buffer() and its component functions are duplicated
from ftobjs.c. The duplication of "CID " table content is just
for the first stage experiment, reusing of existing stream is
expected in proper implementation.

Index: src/sfnt/sfobjs.c
===================================================================
RCS file: /sources/freetype/freetype2/src/sfnt/sfobjs.c,v
retrieving revision 1.133
diff -u -r1.133 sfobjs.c
--- src/sfnt/sfobjs.c   29 Aug 2008 06:09:03 -0000      1.133
+++ src/sfnt/sfobjs.c   29 Aug 2008 09:51:28 -0000
@@ -546,6 +654,82 @@
                              tt_face_lookup_table( face, TTAG_CFF )  != 0 );
 #endif
 
+    /* Fallback for sfnt-wrapped CID or Type1 fonts for Macintosh.
+     * In comparison from CFF OpenType, the essential sfnt tables in
+     * sfnt-wrapped are not documented. For minimum support, we parse
+     * TYP1 or CID table only, other contents are ignored. 
+     */
+    FT_TRACE2(( "has_outline=%d\n", has_outline ));
+    if ( !has_outline )
+    {
+      FT_Library  library = face->root.driver->root.library;
+      FT_Memory   memory = library->memory;
+      FT_Byte*    pstable_data   = NULL;
+      FT_ULong    pstable_length = 0;
+      FT_ULong    ttag_TYP1 = FT_MAKE_TAG( 'T', 'Y', 'P', '1' );
+      FT_ULong    ttag_CID  = FT_MAKE_TAG( 'C', 'I', 'D', ' ' );
+      FT_Bool     is_sfnt_cid = FALSE;
+
+
+      FT_TRACE2(( "check sfnt-PS font.\n" ));
+
+      if ( 0 != tt_face_lookup_table( face, ttag_TYP1 ) )
+      {
+        FT_TRACE2(( "TYP1 table is found in sfnt resource.\n" ));
+        if ( tt_face_goto_table( face, ttag_TYP1, stream, &pstable_length ) )
+          goto Abort_to_load_sfnt_PS;
+      }
+      else if ( 0 != tt_face_lookup_table( face, ttag_CID ) )
+      {
+        FT_TRACE2(( "CID table is found in sfnt resource.\n" ));
+        if ( tt_face_goto_table( face, ttag_CID, stream, &pstable_length ) )
+          goto Abort_to_load_sfnt_PS;
+
+        is_sfnt_cid = TRUE;
+      }
+      else
+        goto Abort_to_load_sfnt_PS;
+
+      FT_TRACE2(( "try to allocate buffer to copy PS table content.\n" ));
+      if ( FT_ALLOC( pstable_data, (FT_Long)pstable_length ) )
+        goto Abort_to_load_sfnt_PS;
+
+      FT_TRACE2(( "copy PS table content to buffer.\n" ));
+      error = FT_Stream_Read( stream, (FT_Byte *)pstable_data, pstable_length 
);
+      if ( !error )
+      {
+        FT_Byte*  p = pstable_data;
+
+
+        FT_TRACE2(( "try to parse PS table in buffer.\n" ));
+        if ( is_sfnt_cid )
+       {
+         FT_Fixed   sfnt_cid_version       = FT_NEXT_ULONG( p );
+         FT_UShort  sfnt_cid_flags         = FT_NEXT_USHORT( p );
+         FT_UShort  sfnt_cid_CIDCount      = FT_NEXT_USHORT( p );
+         FT_ULong   sfnt_cid_CIDFontLength = FT_NEXT_ULONG( p );
+         FT_ULong   sfnt_cid_asciiLength   = FT_NEXT_ULONG( p );
+         FT_ULong   sfnt_cid_binaryLength  = FT_NEXT_ULONG( p );
+         FT_UShort  sfnt_cid_FDCount       = FT_NEXT_USHORT( p );
+       }
+
+        error = open_face_from_buffer( library,
+                                      p,
+                                      pstable_length,
+                                       face_index,
+                                       is_sfnt_cid ? "t1cid" : "type1",
+                                       &face );
+      }
+
+      if ( !error )
+        return error;
+      else
+        FT_FREE( pstable_data );
+
+Abort_to_load_sfnt_PS:
+      ;
+    }
+
     is_apple_sbit = 0;
 
     /* if this font doesn't contain outlines, we try to load */
@@ -499,6 +505,108 @@
   } while ( 0 )
 
 
+  /* Finalizer for a memory stream; gets called by FT_Done_Face().
+ *      It frees the memory it uses. */
+  /* from ftmac.c */
+  static void
+  memory_stream_close( FT_Stream  stream )
+  {
+    FT_Memory  memory = stream->memory;
+
+
+    FT_FREE( stream->base );
+
+    stream->size  = 0;
+    stream->base  = 0;
+    stream->close = 0;
+  }
+
+
+  /* Create a new memory stream from a buffer and a size. */
+  /* from ftmac.c */
+  static FT_Error
+  new_memory_stream( FT_Library           library,
+                     FT_Byte*             base,
+                     FT_ULong             size,
+                     FT_Stream_CloseFunc  close,
+                     FT_Stream           *astream )
+  {
+    FT_Error   error;
+    FT_Memory  memory;
+    FT_Stream  stream;
+
+
+    if ( !library )
+      return FT_Err_Invalid_Library_Handle;
+
+    if ( !base )
+      return FT_Err_Invalid_Argument;
+
+    *astream = 0;
+    memory = library->memory;
+    if ( FT_NEW( stream ) )
+      goto Exit;
+
+    FT_Stream_OpenMemory( stream, base, size );
+
+    stream->close = close;
+
+    *astream = stream;
+
+  Exit:
+    return error;
+  }
+
+
+  /* Create a new FT_Face given a buffer and a driver name. */
+  /* from ftmac.c */
+  static FT_Error
+  open_face_from_buffer( FT_Library   library,
+                         FT_Byte*     base,
+                         FT_ULong     size,
+                         FT_Long      face_index,
+                         const char*  driver_name,
+                         FT_Face     *aface )
+  {
+    FT_Open_Args  args;
+    FT_Error      error;
+    FT_Stream     stream = NULL;
+    FT_Memory     memory = library->memory;
+
+
+    error = new_memory_stream( library,
+                               base,
+                               size,
+                               memory_stream_close,
+                               &stream );
+    if ( error )
+    {
+      FT_FREE( base );
+      return error;
+    }
+
+    args.flags = FT_OPEN_STREAM;
+    args.stream = stream;
+    if ( driver_name )
+    {
+      args.flags = args.flags | FT_OPEN_DRIVER;
+      args.driver = FT_Get_Module( library, driver_name );
+    }
+
+    error = FT_Open_Face( library, &args, face_index, aface );
+
+    if ( error == FT_Err_Ok )
+      (*aface)->face_flags &= ~FT_FACE_FLAG_EXTERNAL_STREAM;
+    else
+    {
+      FT_Stream_Close( stream );
+      FT_FREE( stream );
+    }
+
+    return error;
+  }
+
+
   FT_LOCAL_DEF( FT_Error )
   sfnt_load_face( FT_Stream      stream,
                   TT_Face        face,


##########################################################################

While I was writing the patch in above, I had a few questions:

Q1) There are 2 tag checks in tt_face_init() and sfnt_open_font().
    The purposes would be slightly different, they should not be
    unified?

Q2) For the earliest phase, it would be reasonable to parse
    "CID " or "TYP1" table only and ignore other contents for
    simplicity. Considering such strategy, ftobjs.c would be
    the right place to implement the functionalities to skip
    unrequired parts (as Yamato-san implemented resource fork
    accessing routines). It is not essential to down to TrueType
    driver and jump to CID-keyed font driver.

    But, in next phase, possibly the support of "cmap" table
    would be important (I'm not sure if "ALMX", "ROTA", "HFMX",
    "VFMX" tables are still important when the content of
    "CID " is already parsed). In such case, utilization of
    TrueType font parser would be important.

    I wish I can implement the first phase within 2008, but
    I think I cannot finish the second phase within 2008.
    Can I leave unused "trampolines" in TrueType drivers until
    the second phase? Or, should I restrict the modifications
    in ftobjs.c, until when I start the work for the second
    phase?

Q3) Does anybody have a documentation on "Apple's GX Type1 font"?
    Adobe TN#5180 tells that it was included in Apple Developers
    CD-ROM.


Regards,
mpsuzuki