Re: [PATCH] build: Install pkg-config file for intl

[Eli Schwartz]

On 10/12/22 8:40 PM, Bruno Haible wrote:
>> libtool files are indeed bad, end of story, we all agree. Excellent.
> 
> I didn't say that. But libtool .la files were meant to solve merely two
> problems (dependencies between static libraries, and dependencies between
> shared libraries on some operating systems), and they fall short of solving
> the other problems. So, we don't need to discuss them further here.


:p

libtool .la doesn't even solve these problems either, since they are
only read by libtool, and not even by automake when linking executables.
Other build systems don't even use libtool even for linking libraries.


>> foo-config scripts are indeed bad due to the lack of standardization. I
>> disagree that they are bad due to inability to select 32-bit vs. 64-bit.
>> Historically, one simply invoked foo-config-32 or foo-config-64. And
>> autotools-based build systems can override this by specifying
>> $FOO_CONFIG as the path to whichever config script is desired. It's not
>> elegant, but it does work.
> 
> foo-config-32 or foo-config-64 resurfaces the complexity to the user who
> wants to build a package. Whereas "solving" a problem, IMHO, means to
> deal with the complexity in such a way that it is contained in the
> transparent box.


I did say it's "not elegant, but it does work".


>> In fact, there's nothing *stopping* people from naming them
>> $target-foo-config, and checking for them with AC_CHECK_TARGET_TOOL,
>> except... standardization, or the abject lack thereof.
>>
>> But if people all agreed to name the scripts that way, it would
>> definitely work, wouldn't it?
> 
> Most of the problems of pkg-config would also persist if the general
> approach would be a standardized approach (C).
> 
>> A bigger issue IMO is for portability it needs to work on any OS. POSIX
>> shell scripts don't work on Windows without a lot of fuss -- this means
>> Windows users can't do a good job of finding software utilizing config
>> scripts, so they probably end up using Windows-specific software, maybe
>> even proprietary, as a dependency.
> 
> The "shell scripts don't work on Windows" is IMO solved by Cygwin. Many
> developers on Windows have installed Cygwin and thus have — after setting
> PATH appropriately — the ability to run shell scripts.


And many have not, and/or really want to build MSVC binaries and ship
them to end users, and the most practical way to convince those
developers to use a particular library is if that library doesn't rely
on Cygwin. If shell scripts can be replaced by a key-value file format
that is otherwise equivalent, it is a straight upgrade that helps those
Windows users. :)


> Regarding build systems for packages: Before Cygwin it was customary to
> have two flavours of each script, one in sh syntax and one in .bat or .cmd
> syntax. Assuming Cygwin nowadays, it simplifies maintenance to have the
> script only in sh syntax. This is also the route that the GNU Build System
> has taken: Autoconf-generated configure scripts can be run on Cygwin,
> with CC set to some particular wrapper script (see gettext/INSTALL.windows
> for details).
> 
>> pkg-config has some fake problems listed, such as the inability to find
>> custom .pc files when obviously you very much can -- just like you need
>> to set $PATH in order to find executables in the user home, so too you
>> need to set $PKG_CONFIG_PATH.
> 
> This problem is not fake. It steals actual time and produces actual
> frustration to people who want to build a package that relies on a
> pkg-config, with the usual pkg.m4.


It's not an inability, though! It is an inconvenience. And it's one that
you need to handle *anyway* in order to use the package's executable
utilities after building the package.

Of course, it would be nice to handle both of these problems
automatically. XDG recently added support for recommending ~/.local/bin,
I suppose there's nothing stopping pkg-config from deciding to
automatically handle ~/.local/lib/pkgconfig -- it just needs someone to
propose it, I guess.


>> It also talks about rpath issues, but I don't think this argument makes
>> sense either. Several build systems actually *do* automatically set up
>> rpath based on the locations of shared libraries used in the build, and
>> they don't care if rpath is in the pkg-config file.
> 
> This is interesting, as it could help me solve at least this part of the
> issue. Can you please give me pointers to these "several build systems"?
> Thanks!


Both Meson and CMake automatically do this at build time, Meson's
implementation is at
https://github.com/mesonbuild/meson/blob/master/mesonbuild/backend/backends.py#L731

This allows programs to work while uninstalled.

At a meta level, Meson figures out the path to each shared library used
during the build, and e.g. in the GCC mixin, if that path is not one of
the ones used in `gcc --print-search-dirs` under the "libraries: ="
line, it assumes that that path is needed, and links to

-Wl,-rpath,<the-path> -Wl,-rpath-link,<the-path> <the-path>/libfoo.so

CMake does something similar. Both will by default remove any
automatically calculated build-time rpaths during `make install`, CMake
can control this with `CMAKE_INSTALL_RPATH_USE_LINK_PATH`, Meson doesn't
have an option (perhaps there should be one).

This is a rough heuristic, because `gcc --print-search-dirs` doesn't
necessarily match up to ld.so.conf, which is why it is limited to when
running the programs locally in the build directory (at which point the
primary goal is to make this work seamlessly no matter what, rather than
to comply with distro preferences like "don't have rpaths for default
ld.so directories").


>> The larger issue is
>> that you want rpath for non-default ld.so paths, but not for the default
>> system ones -- but that's a bug in ld.so, as it doesn't have a
>> machine-readable way to determine it.
> 
> Yes, the problem is that it's not documented which library locations
> don't need -rpath. (/usr/lib ? /usr/lib64 ? /usr/lib/amd64 ? /usr/local/lib ?)
> 
> But if the answer was merely retrievable by asking the system configuration
> (ld.so with some option, or such), it would become impossible for one user
> on system XYZ to give their binaries to another user on another machine also
> running system XYZ. For this to work, the set of library locations would
> have to be documented and fixed forever, for each particular system type.


One could also choose to statically link, and make fully shareable
binaries :) this is a common binary sharing approach which usually
relies on building against very old glibcs on ancient CentOS build
servers, but there you have it.

Multiple installations of system XYZ generally don't change their global
ld.so search paths very often, and if they do, this change is rolled out
to all installations at around the same time, so this is not necessarily
as challenging as all that.


>> Then you also mention some so-called flaw of pkg-config, that it depends
>> on pkg.m4, even though it doesn't. This is not even a pkg-config issue,
>> it's an autotools issue, and build systems other than autotools have
>> pkg-config support built in, so it always works.
> 
> You are right that my viewpoint is biased: I care most about the packages
> that use the GNU Build System. Probably the solution to some of the
> problems — from my point of view — is to write a better pkg.m4, similar
> to what you say the other build systems do.


Honestly, the main problem is that pkg.m4 "belongs" to pkg-config rather
than being maintained in the aclocal collection within the automake
source tree. So aclocal cannot guarantee it exists, or *where* it
exists, and you end up having to sync two moving parts.


>> Use of different compilers is an interesting issue to bring up. This
>> mostly works, because the type of flags that typically appear in
>> pkg-config files are in the portable set. As for -pthread, well, it
>> would be neat if there was a pkg-config file describing how to depend on
>> threads.pc, which contained a gcc implementation adding `Libs: -pthread`
>> and an xlc implementation in its own directory (added in tandem with the
>> compiler) specifying something else.
> 
> You can write such a threads.pc by yourself, if you ignore old platforms.
> It depends on the OS and compiler:
>   - ELF platforms: LDFLAGS += -lpthread
>   - AIX, FreeBSD: CFLAGS += -D_THREAD_SAFE
>   - Solaris: CFLAGS += -D_REENTRANT
> Additionally, on AIX, it's best to pass '-qthreaded -qtls' already as part
> of $CC and $CXX.
> 
>> Cross compilation, of course, works perfectly well. Especially if you
>> use pkgconf instead of pkg-config, and can use
>> https://man.voidlinux.org/pkgconf-personality.5
> 
> This is interesting! Is this file format going to be supported also by the
> original pkg-config?


No clue. Most people use pkgconf these days because it's more easily
bootstrapped (it does not depend on glib).

Freedesktop pkg-config currently doesn't even build from source because
it cannot be `autoreconf`ed. This was originally reported in September
of 2020, four months later the project maintainer closed the bug report
-- and several others -- after responding that it can be built from
tarballs by running the included ./configure script. And "pkg-config has
very little active development at this time."

Very little development meaning that the most recent change other than
README or other English prose modifications was in 2019.

pkgconf as the primary maintained implementation is currently interested
in resolving inconsistencies in the format, clarifying behavior, and
extending it with old feature requests such as Cflags.private.


>> Despite this, your gripe is that:
>>
>>
>>> Some people think that a per-target target-pkg-config script is the
>>> solution. But this "solution" works only for cross-compilation
>>> targets defined by the distro, not for cross-compilers that users
>>> have built themselves (by building binutils and gcc with special
>>> configure options).
>>
>>
>> Of course, if one is building binutils and gcc with special configure
>> options, they can create their own per-target target-pkg-config script
>> at the same time they configure GCC and binutils themselves. Why is this
>> not a solution?
> 
> You can call it a solution, yes :-) But it's extra work, most people who
> work with cross-compilers haven't invested. Perhaps because they didn't
> know they would need it? Perhaps because it's not easy (requires hacking
> the pkg-config sources)? Perhaps because there's no tutorial?


pkgconf-personality files aim to solve that by documenting how to define
a cross compiler toolchain, and then invoke either a triplet-prefixed
name as a symlink, or `pkgconf --personality $triplet`.

Still, I think it's worth noting that this is an education and
"convenience tooling" issue, not something that cannot be done if you
know it needs to be done.

Really, I don't think you can get around the need to do *some* work. GCC
could install a pkgconf-personality file, I suppose, but ultimately, the
file needs to know how GCC was configured. It cannot be inferred *just*
from the triplet alone.


>> Disregard everything above, and instead ask a much simpler question.
>> After all the effort you have gone to in describing why you don't think
>> pkg-config is fantastically wonderful...
>>
>> ... what better option are you proposing?
> 
> I'm mostly worried about the integration in the GNU Build System, and
> for non-root users to be able to build packages with as little hassle
> as possible.
> 
> It looks like an improved pkg.m4 will be able to solve some of these
> problems. Thank you for the reference to pkgconf-personality; if
> the improved pkg.m4 parses that file, it would improve the usability
> also in the cross-compilation case.


FWIW, pkgconf when invoked as a symlink named $triplet-pkg-config will
automatically load the $triplet personality file instead of using the
personality that is configured as the default.

So pkg.m4 should work with that out of the box as long as you have:
- a personality file for $triplet
- a symlink for $triplet


>> ... removes libre capabilities from the user.
>>
>> Can you please explain this gravely serious accusation? In what way does
>> adding a pkg-config file to the gettext project cause gettext to become
>> proprietary software, or otherwise software that maliciously restricts
>> what users are permitted to do with their previously Free/Libre and Open
>> Source operating systems?
>>
>> Does it violate the four essential freedoms?
>> Does it track your location?
>> Does it send non-free javascript to the .pc interpreter to be executed?
>> Does it do treacherous computing? Digital Restrictions Management?
>>
>> Please clarify why you think pkg-config files are an enemy of the Free
>> Software movement.
> 
> The .pc files invite people to use the current pkg.m4 (rather than the
> gettext.m4 from the GNU gettext package).
> 
> While gettext.m4 makes it easy to use to use a libintl that was installed
> by the user with e.g. --prefix=$HOME/installed-gettext, pkg.m4 makes it
> hard and frustrating. The net effect is that users struggle with that,
> and after two or three failures abandon and say "oh well, let me just use
> what my distro vendor provided".
> 
> Whereas the idea of Free Software is not only that it *technically possible*
> to modify the source code and build and use that modified source code, but
> also that, as a user, you are *invited* to do so.


So you're making a distinction that pkg-config "encourages" people to
not exercise their freedoms because it is complicated to exercise their
freedoms.
But not that pkg-config removes those freedoms (e.g. from power users).

Truthfully, I could argue that GNU autotools discourages people from
exercising their freedoms, because it is too difficult to use on Windows
with MSVC, so Windows users do not feel invited to do so.

I don't think attempting to determine this sort of thing is particularly
fruitful, it's way too opinionated and one person's invitation is
another person's turnoff.

I would just look at whether the authors or maintainers of the source
code are friendly towards people attempting to do so, and open to
implementing suggestions for making that easier, and generally give off
those "good-faith attempt to be inviting" impressions.

If they do, then you can qualify the project as "fulfilling both the
letter and spirit of Free Software".

If they don't, then you can qualify the project as "fulfilling the
letter of Free Software, but not the spirit".

...

Bringing this back to pkg-config, autotools *.m4 files cannot be used in
build systems other than autotools, while pkg-config files can be used
by any build system. So they are usually advertised as "the friendly and
inviting way to allow anyone to use my software, regardless of OS or
build system". And most people using pkg-config believe they are doing
so in order to be welcoming, and usually don't distribute *.m4 files at all.

I would argue that in most cases, pkg-config *is* the preferred method.
Gettext is actually an outlier, because sometimes it is e.g. baked into
the libc.

Meson actually handles this by providing a dependency interface that
knows how to treat pkg-config as one of the lookup methods, but not the
only one. Most software libraries only need pkg-config, though.

For intl, it's implemented here in python:

https://github.com/mesonbuild/meson/blob/7912901accaee714fc86febdc72f4347b9397759/mesonbuild/dependencies/misc.py#L469-L515

We could of course add a pkg-config lookup to this as well. Currently we
first check if it resolves as a "builtin" lookup by attempting to link
to symbols in libc, and then we do some manual scraping for checked libs
and has_header. There may very well be improvements to be made, but
having a pkg-config file would mean we don't need to reimplement some
gettext.m4 logic that basically just checks for certain OSes and adds
some flags when figuring out exactly how to link to the externally
installed intl library.


-- 
Eli Schwartz