Re: AC_SEARCH_LIBS and non-cdecl calling conventions

[John Calcote]

On 10/27/2009 11:44 AM, Matěj Týč wrote:
> On Mon, 2009-10-26 at 21:34 +0100, Ralf Wildenhues wrote:
>
>    
> > 3) The pressing issue that Matěj complained about was when configure
> > fails to detect all libraries, because all of them have a different
> > calling convention.  Right?
> >      
> The situation that I have encountered the case of cross-compilation.
> One library has a different library name for *nix version and respective
> Windows version and the Windows one uses the stdcall (at least I think
> so) calling convention.
> And when one thinks about that this should be a quite common problem -
> that is when you cross-compile, you may have some difficulties while
> checking for your libraries.
>
> I don't really understand that calling conventions - related stuff, so I
> can't say anything highly intelligent to the other points.
>
> Maybe just to remind that people sometimes complain about the configure
> script slowness, so we should be careful when doing things that could
> make the script's exec time even longer.
>    

There are literally dozens and dozens of calling conventions. Each 
processor architecture specifies one or more calling conventions, based 
on hardware capabilities and features. In fact, there are over a dozen 
calling conventions on Intel x86 hardware alone. Symbol name mangling is 
closely related to calling convention, but not actually part of a 
calling convention definition. The relationship between calling 
convention and name mangling is defined by the application binary 
interface (ABI).

On Intel hardware, symbol name mangling includes the addition of one or 
more underscore characters to either end of the name, upper- or 
lower-casing of the symbol name, appending an ascii decimal string 
representing the number of stack words of parameters passed, the use of 
additional symbols to separate these bits of data (e.g., the at-sign 
'@'), etc.

You might think, based on this information, that it's by sheer luck that 
we've had so few problems so far with AC_CHECK/SEARCH_LIB(S), but the 
fact is, the designers of these macros were smarter than that. By 
attempting to import a symbol using a common prototype [char * 
symbol();], they've managed to eliminate all calling convention issues 
except those where the number and/or types of parameters enter into 
determining the name.

Which brings us to the issue at hand: To obtain a proper link symbol 
within name mangling schemes that *do* care about number and types of 
parameters, we have to provide a more specific function signature to the 
macro. This signature has not only to specify the function parameter 
types, but also any compiler-specific attributes necessary to indicate 
the calling convention used (on platforms that provide more than one 
calling convention - like Windows).

Passing a fully-specified function prototype in the macro is a 
reasonable thing to do for the sake of various C++ mangling schemes, as 
well as for the sake of other calling conventions that require the 
number and/or types of parameters to determine the proper mangled symbol 
name. However, passing compiler-specific calling-convention attributes 
(e.g., __attribute(stdcall), __stdcall, __declspec(stdcall), __fastcall, 
__cdecl, etc.,) will clearly not work, because they're 
compiler-specific. Each compiler (even on the same system!) uses 
different such keywords to specify the same symbol attributes.

Ideas?

John