Re: build system: devpdf/download regression

[Ingo Schwarze]

Hi Branden,

G. Branden Robinson wrote on Sat, Jun 18, 2022 at 03:06:48PM -0500:

> I did not consider, and did not test, the scenario where the build and
> deployment systems differ, and the latter was more capable than the
> former.

I see.  Actually, that's a typical situation in OpenBSD ports builds.
For many (if not most) ports, build dependencies and run dependencies
differ, and it isn't even unusual that none of them is a superset
of the other.  At build time, you cannot rely on having access to
mere run-time dependencies; at run-time, you cannot rely on having
access to mere build dependencies.  I would expect that many other
packaging systems employ similar concepts.

On OpenBSD, ghostscript is neither a build dependency nor a run
dependency of groff.  But if the end-user installs it, they can
can use it at run time, including with groff, and i suspect most
users do that.  On OpenBSD, no special marking is needed in such
a situation, but i believe some other packaging systems call this
kind of thing an "opional dependency" or a "recommended package".


> At 2022-06-18T19:16:10+0200, Ingo Schwarze wrote:

>> On a tangent, i suspect similar problems may be hiding elsewhere
>> that will bite people who disable (or lack) other features in their 
>> system that i left enabled.  That is, some integral parts of
>> groff only being built because some optional stuff (like building
>> documentation in an optional format) depends on them.
>> But auditing for that kind of problem is hard because there are
>> so many *.am files scattered around the tree and the automake
>> infrastructure involved makes reading them a non-trivial task.

> It's not _too_ hard because as a rule we've controlled the number of
> places where this happens.  The main variables are (1) the generated
> formats of our Texinfo manual, which I've mainly foreclosed by shipping
> them all with the distribution archive; (2) the PDF output driver, as
> you note; and (3) the HTML output driver, which similarly depends on
> device and font description files to be generated at build time but,
> even if they are, may not be able to run successfully because of a lack
> of support tooling on the build host.
> 
> I think these are superable problems.

Nice, so it appears you audited for other instances of similar
problems, too.  Thanks for doing that!


>> The quick and dirty patch appended below fixes the immediate problem
>> in my partciular setup.  Now all the expected devpdf files get
>> built and installed, including the "download" file and the individual
>> font description files like TR, TB, CR and so on.

> Importantly, this should ship the U-* font descriptions if they are
> present and not if they aren't.

That's bad because it might result in erratic packaging behaviour,
namely different package content depending on what happens to be
unstalled at package build time.  So i fear i'll have to investigate.

> This does pose a difficulty for your build scenario; you must decide
> whether you want to cater to users wishing to run gropdf using the URW
> fonts, or not.

So far, i never cared to educate myself what "URW fonts" even are,
or what they might be useful for.  I know there have repeated been
messages about those fonts on this list, but i never bothered to
read them.

> If so, you will need to add a build dependency on them.

Hum.  None of these files exist anywhere in the OpenBSD ports tree:

  URWGothic-Book.t1 URWGothic-Book.pfb URWGothicL-Book.pfb

The following file does exist in the print/ghostscript/gnu-fonts port:

  /usr/local/share/fonts/ghostscript/a010013l.pfb

Even when the port is installed, building groff from git does not find
the file because /usr/local/share/fonts/ghostscript/ is not in the gs(1)
search path (at least not as reported by gs -h).  When i add

  --with-urw-fonts-dir=/usr/local/share/fonts/ghostscript

to the ../configure invocation, these lines change in ../configure
output:

  -checking for URW fonts in Type 1/PFB format... none found
  +checking for URW fonts in Type 1/PFB format... \
        found in /usr/local/share/fonts/

  - use URW fonts for PDF output     : no
  + use URW fonts for PDF output     : yes

  -configure: URW fonts in Type 1/PFB format were not found.
  -[... long paragraph ...]

But then i get this at build time:

  +BuildFoundries: warning: line 33: Unable to locate font(s) 
URWGothic-Demi.t1,URWGothic-Demi,URWGothicL-Demi,a010015l.pfb
  +BuildFoundries: warning: line 34: Unable to locate font(s) 
URWGothic-DemiOblique.t1,URWGothic-DemiOblique,URWGothicL-DemiObli,a010035l.pfb
  +BuildFoundries: warning: line 35: Unable to locate font(s) 
URWGothic-BookOblique.t1,URWGothic-BookOblique,URWGothicL-BookObli,a010033l.pfb
  [...]

and many more similar messages, and then:

  +BuildFoundries: warning: line 75: 
  +The path(s) used for searching:
  +ARRAY(0x23cef899970)

All the same, later, there is also

  +BuildFoundries: notice: Generated U-AB...
  +BuildFoundries: notice: Generated U-ABI...
  +BuildFoundries: notice: Generated U-AI...
  [...]

and many similar messages, intermixed with many messages like the following:

  +afmtodit: both Tcommaaccent and uni0162 map to u0054_0327 \
        at /usr/local/bin/afmtodit line 6555.
  +afmtodit: both tcommaaccent and uni0163 map to u0074_0327 \
        at /usr/local/bin/afmtodit line 6555.

Apparently, nothing changes during "make dist".


Then still later, at port build time, if i add a build dependency
on print/ghostscript/gnu-fonts (which might be acceptable because it
has no dependencies) and also add

  CONFIGURE_ARGS += --with-urw-fonts-dir=/usr/local/share/fonts/ghostscript

to the port make file, the line

   use URW fonts for PDF output     : no

in ./configure output does *not* change,
but i get the "Unable to locate font(s)" messages mentioned above and
the weird "ARRAY(0x23cef899970)" message anyway, but *not* the
"Generated U-*" and "commaaccent" messages, and the set of installed
files does *not* change compared to running without --with-urw-fonts-dir.

I don't have the slightest idea whether any of this is expected or wrong...

> If not, those users will have a tough time until we can get user-facing
> tooling (the infamous "install-font" problem) fully developed and
> supported.  The short path is probably for them to build groff from
> source and reënable some of the stuff you're turning off.

That doesn't seem simple at all, even i didn't really figure out so
far what it might be that they would have to do at build time...

Besides, we strongly discourage installing anything on OpenBSD without
a port, so that would be a really bad option.  Even if a user wants
to install something just for themselves, we recommend they build a
port, even if they don't publish it.

Do you think anybody might actually want to have these fonts?
So far, i don't recall anyone asking about them on OpenBSD lists.
If the answer is "yes", i might have to talk to one of our developers
who specialize in porting fonts whether they can prepare and commit
a well-designed port of the URW fonts and commit that (of course,
that port must not have any build dependencies).  Then i could make
our groff port depend on that URW fonts port with the next groff
update.
If the answer is "no" then i guess i need to find a way to reliably
disable all the URW stuff such that it doesn't accidentally get
triggered depending on what someone might happen to have installed,
ideally also reducing the considerable noise that BuildFoundries
spews at build time.


> Your patch seems valid and minimally intrusive to me; it may look a
> little bit weird (for an Automake file), but the circumstances leading
> to it are weird.  Normally I think Automake would require us to store
> the names of the files to be installed in one of its magical variables,
> but that's necessary here because our Automake file _already_ goes
> behind its back with an install hook and a `for` loop.

Thanks for checking; for now, i pushed it.
When you find a better way, you are welcome to polish it futher!

Yours,
  Ingo


>> diff --git a/font/devpdf/devpdf.am b/font/devpdf/devpdf.am
>> index 363d4cdd0..64ed657d4 100644
>> --- a/font/devpdf/devpdf.am
>> +++ b/font/devpdf/devpdf.am
>> @@ -51,6 +51,8 @@ EXTRA_DIST += \
>>    font/devpdf/Foundry.in \
>>    font/devpdf/util/BuildFoundries.pl
>>  
>> +all: font/devpdf/stamp
>> +
>>  font/devpdf/enc/text.enc:
>>      $(AM_V_at)$(MKDIR_P) $(top_builddir)/font/devpdf/enc
>>      $(AM_V_at)cp -f $(top_srcdir)/font/devps/text.enc \