Re: [ft-devel] Known problem with FT_StreamRec::base being non-NULL also for file-based streams?

[mpsuzuki]

On Mon, 01 Mar 2010 22:29:18 -0500
Behdad Esfahbod <address@hidden> wrote:

>On 03/01/2010 09:18 PM, address@hidden wrote:
>> How about using
>> 
>>     if ( face->stream->read == NULL )
>> 
>> instead of
>> 
>>     if ( face->stream->base != NULL )
>
>Yes, that's what I'm planning to do instead.

Thanks!

>Checking the source code, I wonder if I should also check for (face_flags &
>FT_FACE_FLAG_EXTERNAL_STREAM) to detect whether it's an mmapped stream or the
>user provided it (and hence we cannot mprotect).  The docs say: "Don't read or
>test this flag."

Please let me know more detail about what the information
is needed at the part. Yet I'm not understanding the idea
of "blob" in HarfBuzz.

Although the name FT_FACE_FLAG_EXTERNAL_STREAM looks like as
if it is fixed just before the creation of FT_Face object,
sometimes FreeType2 copies the (partial) content of the buffer
passed by the client to internally allocated buffer, and
clears the flag.

In builds/unix/ftsystem.c, if mmap() for specified file is
failed, FT2 allocates the buffer and copies the content of the
file. In both scenarios (mmap() ok, and mmap() not ok),
FT_FACE_FLAG_EXTERNAL_STREAM is not changed. So it is impossible
to distinguish whether ( FT_FACE_FLAG_EXTERNAL_STREAM == 0 )
and ( read == NULL ) is mmap()-ed or malloc()-ed stream.

# Worse scenarios would be found for MacOS-specific font.
# If FT2 client make a memory image of sfnt-wrapped PS Type1
# and pass it to FT2, FT2 allocates new buffer internally 
# and copies TYP1 table only. Then FT_FACE_FLAG_EXTERNAL_STREAM
# is turned off.

I think the request of info about the origin of font memory
image is reasonable, but current FT2 is designed to provide
an abstract FT_Stream object and does not expect FT2 clients
to touch the internal of FT_Stream, even some internals are
exposed. As you know, FT2 clients can receive FT_Stream object,
but cannot create FT_Stream object by public API.

# I'm not sure if 2 public APIs FT_Stream_OpenLZW() and
# FT_Stream_OpenGzip() are useful for FT2 clients.

To guarantee the memory buffer is obtained by mmap() in FT2,
including previous releases, the most stable way would be
mmap() in FT2 client and pass the memory image to FT2. 

If you don't want to such client-side mmap(), it is possible
to detect if the stream is mmap()-ed, but the code would be
quite complexed.

--------------------------------------------------
1) create a temporary writable font.

2) open the temporary font.

3) to assure the stream is memory-based,
   check FT_Stream->read is NULL.

4) to assure the memory is mmap()-ed,
   modify a content of temporary font and
   check the change is reflected to memory image.

5) memorize the address of FT_Stream->close, as
   the address of ft_close_stream_by_munmap().
   
6) destroy FT_Face object and erase the temporal
   font.
--------------------------------------------------

I think HarfBuzz developers don't want to insert such ugly
code.

In summary, it is quite difficult to check if the stream
is mmap()-ed by existing releases of FT2. If new API is
added in future release of FT2, does it help HarfBuzz?

Regards,
mpsuzuki

P.S.
I've ever worked for a sample implementation of new service
getting the pathname from FT_Stream. Similar method can be
applicable to provide a method to check if the memory of
the stream is internally mmap()-ed.