bug#44462: Problem with get_multilibs on macOS

[Jacob Bachmeyer]

Fred Wright wrote:
> On Sun, 8 Nov 2020, Jacob Bachmeyer wrote:
> > Fred Wright wrote:
> > > On Thu, 5 Nov 2020, Jacob Bachmeyer wrote:
> > > > Fred Wright wrote:
> > > >
> > > > > I traced the results of get_multilibs (as used by the libffi 
> > > > > tests) in many macOS versions, and even the "successful" cases 
> > > > > seem to have questionably useful results:
> > > > >
> > > > > [...]
> > > > >
> > > > > The "/usr/." result returned on 10.4, 10.5, and 32-bit 10.6 is the 
> > > > > same as what I see on Ubuntu 14.04, CentOS 7, and Fedora 25, 
> > > > > though it's not clear to me what it's supposed to represent.
> > > >
> > > > I have a suspicion that this feature is designed to support testing 
> > > > with an "experimental" compiler build that is not installed on the 
> > > > system and may be useless with system compilers generally, or with 
> > > > Apple's compilers specifically, if Apple does not use multilib.
> > >
> > > Apple supports the concept of multi-architecture binaries, but not 
> > > in the same way that multilib does it (AFAIK).  Macs can have 
> > > "universal" binaries, which are archives combining multiple 
> > > per-architecture slices. This is applicable to object files, shared 
> > > libraries, and executables.  If
> > > the build setup allows it, it can be as simple as including multiple 
> > > architecture options in the compile command.  E.g.:
> > >
> > >     cc -arch x86_64 -arch i386 -o hello hello.c
> > >
> > > Under the hood, the compiler driver runs a separate compile/assemble 
> > > for each architecture, and then combines the object files.  The 
> > > linker supports universal binaries directly.
> > >
> > > With this arrangement, architecture-related conditionals in the 
> > > source code work just fine, but what *doesn't* work is having 
> > > architecture-related parameters in a configure script, which is 
> > > unfortunately not as uncommon as it ought to be.
> >
> > No, Autoconf pushes programs to respond to features instead of using 
> > known architectures.  This approach has been very successful:  as I 
> > understand, most programs using Autoconf needed no changes at all to 
> > port to RISC-V even if they were written long before RISC-V existed.  
> > Many programs, if they had previously been ported to any 64-bit 
> > architecture, needed no changes at all for x86_64.  Autoconf has 
> > achieved something architecture #ifdefs cannot do: provide automatic 
> > portability to a new architecture that did not exist when the program 
> > was written.  I consider Autoconf's approach here completely vindicated.
>
> I'm not talking about checking for specific architectures; this is 
> about checking architecture-related *properties*.  For example, a 
> configure check for sizeof(long) is incompatible with 
> multi-architecture builds, while using LONG_BIT works just fine.  But 
> there are build procedures that do things like the former, for no good 
> reason.

As far as I can tell, LONG_BIT as a preprocessor symbol is relatively 
new; newer than many programs that have demonstrated this level of 
Autoconf-assisted portability.  On older systems, glibc provides 
CHAR_BIT and you can find the bit-length with CHAR_BIT*sizeof(long).

> > > > > Since get_multilibs already has code to return an empty string in 
> > > > > the "remote" case (where it assumes this function won't work), I 
> > > > > just added code to unix.exp to set multitop to "" for all "darwin" 
> > > > > targets, thereby short-circuiting almost all og get_multilibs.  
> > > > > That certainly fixes the problem with the libffi tests, and 
> > > > > doesn't change any non-Mac behavior, though I don't know if that's 
> > > > > the ideal fix.  The whole get_mutilibs function looks pretty ugly 
> > > > > anyway, and it's generally recognized that relying on -dumpspecs 
> > > > > is a bad idea.
> > > >
> > > > It is most certainly not ideal.  A better solution is probably to 
> > > > add a test to get_multilibs to return an empty string if the 
> > > > compiler is not GCC.  Of course, if another compiler pretends to be 
> > > > GCC enough to pass that check, but does not actually implement 
> > > > -dumpspecs, that is not our bug.
> > >
> > > [...]
> > > The libffi test suite comes up with a "compiler_vendor" variable 
> > > which seems to be able to distinguish clang from gcc, though I 
> > > haven't looked at the details.
> > >
> > > Fixing get_multilibs properly would probably mean making it both 
> > > highly platform-specific and optional.
> >
> > The get_multilibs procedure is *not* platform-specific; it is 
> > GCC-specific. I am still unsure how exactly it is used.
>
> Since it seems to involve file paths, it may be specific to 
> combinations of GCC and platforms.
>
> But since neither of us seems to know very much about what 
> get_multilibs is trying to do, it's hard to discuss it intelligently. :-)

This is the main reason that get_multilibs failing on MacOS X will be a 
known bug in 1.6.3.

> > > > This issue on Mac OS X will probably be a known bug in 1.6.3 and 
> > > > fixed in 1.6.4.
> > > >
> > > > > I primarily tested my patch against the 1.6.2 release, since the 
> > > > > current master won't install from a non-git directory, and also 
> > > > > has multiple failures in its own tests (even on Linux).  The patch 
> > > > > is nearly identical between the 1.6.2 and master cases, anyway.
> > > >
> > > > Are we looking at the same current master?  I have commit 
> > > > 3d62df24deedfb3c7c3e396a31b8ce431138eb49 here and all of the tests 
> > > > pass.
> > > >
> > > > ****These other problems are potential release blockers for 1.6.3.****
> > > > Can you file another bug report with the test failures and more 
> > > > information about these issues?
> > >
> > > I looked into this more closely and it's probably related to the 
> > > non-git issue.  When running from a non-git directory, the configure 
> > > script reports a "fatal" error, but then goes on to complete with a 
> > > zero exit status and a more or less buildable setup, so you have to 
> > > be paying close attention to the output to notice.
>
> Actually it now looks like the two things are unrelated; I filed two 
> separate bugs.

Thank you; PR44544 should now be fixed on master; I have responded 
accordingly at that report also.  I am still looking into PR44545.

> > > > > I can send the current patch, either as a bare email or as an 
> > > > > attachment. AFAIK, Savannah doesn't have the pull request / merge 
> > > > > request concept.
> > > >
> > > > This will need to be fixed in libgloss.exp, not unix.exp.  I am 
> > > > putting my foot down on fixing bugs in DejaGnu's own tree directly 
> > > > instead of hacking around them like that.
> > >
> > > Well, OK, but there seem to be other similar hacks in unix.exp, and 
> > > if the idea is that get_multilibs is completely useless on the Mac 
> > > (which appears to be the case with the current implemenation, 
> > > anyway), then disabling it in the target-related code doesn't seem 
> > > unreasonable.
> >
> > It is not completely useless even on MacOS X -- a user could install 
> > GCC and expect a testsuite to use it, particularly in the case of a 
> > cross-compiler for embedded development.
>
> If it's a cross compiler, then by definition it can't run the 
> resulting code on the host platform.  But since get_multilibs already 
> excludes all remote cases, it wouldn't be able to run it on a separate 
> target platform, either.
>
> Besides, aren't files like baseboards/unix.exp based on the *target* 
> platform, not the host?  If so, then it seems like disabling 
> get_multilibs for the Mac there is exactly the right thing, at least 
> until such time as get_multilibs can behave usefully for a Mac target.

Untangling cross-development configurations is an ongoing project.

GCC could (in theory) be installed on any platform.  A user may build a 
native GCC for MacOS X and want to test or use it.  In short, the 
solution needs to test the actual issue -- is $compiler GCC? -- instead 
of assuming that it is or is not based on the platform.


-- Jacob