Re: ftdump can show the CID registry, ordering, and supplement?

[suzuki toshiya]

Dear Alexei,

Excuse me; please let me ask your background to scan
all CIDs. Is it a confirmation of the existence of
the glyph description for a given CID instead of
the declaration of the CID, as fontconfig does?

Currently, my patch cares only FT_Face->num_glyphs
for the face loaded by the t1cid driver. Do you
concern as "the num_glyphs is no more than the
declaration, there is no guarantee that the user
can get a glyph image of the CID within the range
0...FT_Face->num_glyph, FT_Load_Glyph() test is reliable"?

If so, the testing code might be like this:

https://gitlab.freedesktop.org/mpsuzuki/ft2demos-mps-wip/-/compare/mr-ftdump-print-cid-ros-assume-order...ftdump-validate-all-cid?from_project_id=20389

When I execute it to sample CID-keyed fonts
(on O'Reilly's website) and other OpenType/CFF,
it is slow but acceptable as an optional feature.

But the problem I found is that the sample CID-keyed
fonts have many unimplemented CIDs; the font declares
them, but FreeType fails to load their glyph
instructions. If the FreeType library is built
with the FT_TRACE option, these sample CID-keyed
fonts on O'Reilly website generate many warnings, like:

/CIDSystemInfo dictionary
   Registry:            Adobe
   Ordering:            Japan1
   Supplement:          1

   Implemented CIDs:
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=1
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=2
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=3
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=4
(...)
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=8355
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=8356
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=8357
cid_load_glyph: invalid glyph stream offsets
could not load a glyph for CID=8358
     0,422,425-500,629,633-740,780-7477,7479-7554,8229-8263,8284-8285

Regards,
mpsuzuki

On 2023/04/21 5:08, Alexei Podtelezhnikov wrote:
> Hi Suzuki,
>
> Using these fonts
> (https://resources.oreilly.com/examples/9781565922242/tree/master/adobe/samples)
> I learned that FT_Load_Glyph interprets the glyph_index as CID. Could
> you scan the CID range directly using FT_LOAD_NO_SCALE? Or, is this
> still too slow?
>
> Thanks
> Alexei
>
> On Thu, Apr 20, 2023 at 12:47 AM suzuki toshiya
> <mpsuzuki@hiroshima-u.ac.jp> wrote:
> > Dear Alexei, Werner,
> >
> > Here is reworked version, which assumes
> > "no zig-zag mapping is found by FT_Get_CID_From_Glyph_Index()".
> >
> > https://gitlab.freedesktop.org/mpsuzuki/ft2demos-mps-wip/-/commit/453de7aa3f9ccd54824ab3468c5a3b91507a86a9
> >
> > ( 
> > https://gitlab.freedesktop.org/mpsuzuki/ft2demos-mps-wip/-/commits/mr-ftdump-print-cid-ros-assume-order
> >  )
> >
> >
> > Regards,
> > mpsuzuki
> >
> >
> > On 2023/04/20 12:46, suzuki toshiya wrote:
> > > Dear Alexei,
> > >
> > > Umm, TN#5092 is a document in the world of pure CID-keyed font,
> > > so it does not care about the mapping between the CID and
> > > TrueType (or CFF) GID, so I'm unsure if it is a solid footstone
> > > to exclude the unordered CIDs.
> > > For example, "GB1 represents the first version of the ordering
> > > based on the Chinese GB 2312-80 character set" describes
> > > "how CID numbers are assigned to the characters? It is based
> > > on the order of GB 2312-80", it might be independent with
> > > the internal of CID-keyed font.
> > >
> > > But anyway, yet I have no concrete example to justify my
> > > concern about "zig-zag" mapping between GID and CID.
> > >
> > > OK, so I propose - I'll rework my patch based on the assumption
> > > "the larger GID is mapped to the larger CID, no zig-zag
> > > mapping from GID to CID", to avoid 64kByte array allocation.
> > > Then, when ftdump receives a font breaking this assumption,
> > > ftdump aborts with an error, and indicates an email address
> > > (or website?) to report the issue. Is it acceptable workaround?
> > >
> > > Regards,
> > > mpsuzuki
> > >
> > > On 2023/04/20 11:08, Alexei Podtelezhnikov wrote:
> > > > Dear Suzuki,
> > > >
> > > > > Indeed. If GID-CIDs are already sorted, it's easy to dump the
> > > > > available CIDs in a compressed syntax. Although I have no sample
> > > > > font including unsorted CIDs, I'm unsure whether it is required
> > > > > in some specifications.
> > > >
> > > > The order of CIDs is in "registry, *ordering*, and supplement". Also
> > > > it is mentioned in the specs (5092): "A character collection consists
> > > > of an ordered set of all characters". I think we can rely on that.
> > > >
> > > > If there is a CID to glyph index mapping in each file, why doesn't
> > > > FreeTyope generate a cmap?
> > > >
> > > > Thanks,
> > > > Alexei