Re: [ft-devel] Known problem with FT_StreamRec::base being non-NULL also for file-based streams?

[Behdad Esfahbod]

On 03/11/2010 11:27 AM, address@hidden wrote:

> Thanks. I understand the info needed by HB is whether HB can
> modify the memory image without duplication, or HB should copy
> the memory image before duplication.

Correct.


> I and Werner agreed that the easiest way to guarantee the
> origin of buffer as mmap()ed or malloc()ed is the font
> image preparation in HB/Pango side.

Well, I can see how that argument is made.  Still, I'd rather hb-ft be able to
detect whatever it needs from the given FT_Face directly.

Plus, the face is not created by Pango.  It's created by cairo.  Being able to
pass faces around libraries freely without worrying about how it was created
makes things much easier.


> But, taking a glance
> on Pango, I guess, there might be some delay between the
> invocation of FT_New_Face() and HB blob creation.
> The duplication of unwritable font image to writable buffer
> occurs for all faces? Or, the duplication occurs when the
> first modification is tried (to fix OpenType bug in runtime)?

On first modification.  It's very far away (design wise and timeline wise)
from when the face was created.


> If latter scenario is correct - when Pango is going to
> create FT_Face object, Pango cannot know if the duplication
> will occur in future, so, my proposal (HB/Pango side font
> image preparation) will cause unwanted memory consumation
> by loading all faces to writable memory. It won't be good
> idea.
> 
> # In Pango library, when PangoFT2Font->face is created once,
> # it should not be changed anymore? If replacing the face is
> # permitted, I want to create the earliest unwritable face
> # from mmap()ed image, then replace it by writable face with
> # malloc()-and-read() image when Pango/HB tries to modify it.
> # Pango has an API to expose PangoFT2Font->face to the client
> # (pango_ft2_font_get_face()), but it is classified as deprecated
> # interface. I wish Pango library is changing to hide raw
> # FT_Face object from Pango client.

PangoFT2 is deprecated completely.  My main concern is pangocairo and pangofc.


>> This fails for examples when:
>>
>>  - Font data is in ROM.  In this case mprotect() will fail and harfbuzz will
>> make a copy of the memory.  Not a huge problem.
> 
> Indeed. If FT2 could mmap() readonly font file successfully,
> mprotect() will fail.

We mprotect() PRIVATE, so it wouldn't fail even if the font file is readonly.
 But if the memory area is readonly (as in ROM in an embedded device), then we
fail.  Or maybe mprotect() passes but trying the modification segfaults.  But
we don't have to worry about those non-standard cases right now.


>>  - FreeType malloc()ed the font data.  In this case, mprotect() is not
>> necessary and will probably affect memory beyond the font data (since 
>> mprotect
>> works on whole pages).
> 
> Umm, I think, mprotect() for malloc()ed memory causes
> undefined result.
>
> http://www.opengroup.org/onlinepubs/000095399/functions/mprotect.html
>> The behavior of this function is unspecified
>> if the mapping was not established by a call to mmap().
> 
> To avoid such ambiguity, we should know if the buffer
> is mmap()ed or malloc()ed, before mprotect() - am I
> misunderstanding?

You're right.  It works on Linux, but unspecified generally.


>>  - Font data is coming from the user.  In this case it may not be desirable
>> to modify the data.
> 
> Indeed. Could you tell me which function is used to push
> user-provided font data? Is it in cairo layer?

hb-ft takes an arbitrary FT_Face.  So does cairo.  It cannot happen with pango
right now, but pango is not the only use case I have to design for (I wish it
was!).


>> Adding API to FTStream to be able to detect the above cases, specially the
>> user-provided data, would be useful.
> 
> Again thank you for comment about the additon of new API.
> 
> As I've sketched, it is possible to get detailed info of
> FT_Stream object. My current sketch is huge, and I have
> a few issues to be discussed for further improvement of
> FT_Stream object.

Something along the lines of what you have sketched to get the allocator used
is useful, yes.


> Another idea is an addition of the arguments for FT_Open_Face(),
> to specify 3 scenarios for font loading.
> 
> 1) only mmap() is tried.
> 2) only malloc() + read() is tried.
> 3) mmap() is tried, then, malloc() + read() is tried (current behaviour)
> 
> By using 1) and 2), HB/Pango can distinguish the buffer is
> mmap()ed or malloc()ed exactly. I will post a patch for FT2
> and Pango for further discussion. Please wait a few days...

Not a huge fan for reasons mentioned above.

Thanks,
behdad

> Regards,
> mpsuzuki
>