[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: libltdl exports no symbols (cygwin)

From: Charles Wilson
Subject: Re: libltdl exports no symbols (cygwin)
Date: Wed, 31 Jan 2007 01:42:50 -0500
User-agent: Thunderbird (Windows/20061207)

Eric Blake-1 wrote:
I am a bit wary that it may cause regressions in users' packages, as
they may be using LT_SCOPE.  But I see that with the other change, this
is needed.
User-package use of internal libtool macros (e.g. LT_SCOPE) is outside libtool's control.
Hmm, yes.  I've never been certain whether LT_SCOPE was *intended* to be
purely internal, or for the user to use.  The latter seems inappropriate
to me now, but I don't think people were aware of this all the time.

Indeed - ever since this patch, CVS M4 is now broken on cygwin.

Err...and this is news? <snark>For over three years, CVS M4 has had serious issues on cygwin. Every six months or so, I used to give it a go, but finally gave up -- as I have said before, making M4 depend upon the dustiest corners of libtool/libltdl functionality was a serious misdesign for such a fundamental tool. You can't have a working m4 2.0 unless libtool/libltdl on your platform fully supports modules, in both compiled-in (dlpreopen) and externally loaded form (dlopen/LoadLibrary). But without m4 2.0, you won't have working autoconf-NEXT-MAJOR-VERSION -- which may or may not be required for the libtool version that actually supports modules on your target platform as described above. Words fail me...</snark>

 I am
all sorts of link errors along the lines of:

libtool: link: gcc -shared  modules/.libs/gnu.o   m4/.libs/libm4.dll.a    -o
modules/.libs/gnu-0.dll -Wl,--enable-auto-image-base -Xlinker --out-implib
-Xlinker modules/.libs/gnu.dll.a
modules/.libs/gnu.o: In function `m4_regexp_compile':
/home/eblake/m4-head/modules/gnu.c:140: undefined reference to

This is because M4 was relying on auto-imports, and now that libltdl
has an explicit export, the auto-import mechanism is disabled.

Let's be very precise with terminology here. M4 may or may not be relying on auto-imports -- but that is beside the issue. The problem is that auto-EXPORT when linking (e.g. creating) the DLL (in this case, m4.dll) is turned off when any .o or .a(e.g. convenience archive) included in the link contains symbols which are explicitly exported.

In this case, when building the libm4 module, DLL_EXPORT is defined -- which triggers the LT_SCOPE == declspec(dllexport). This is bad, since we expect that the libltdl convenience archive contains _foo, not __imp__foo. [[ However, it appears that the "convenience" archive actually contains declspec(dllexport)-decorated symbols...this is because convenience archives are compiled with the same options as the eventual target: in this case, the m4 dll. It is "pic", so the libltdl convenience archive is "pic" -- which in this context means declspec(dllexport). ]]

The simplest workaround for CVS M4 is to '#define LT_SCOPE extern' before including ltdl.h when building libm4 DLL, AND to do the same when building the libltdl library *as a convenience archive*. This will ensure that the symbols in libltdl are not decorated, and that none of the components linked together to create libm4 DLL contain dllexport-ed symbols; thus re-enabling auto-EXPORT behavior when linking the m4 dll.

I still think the fix to libtool was the correct thing to do, but in order
to fix M4, I need some help.  It looks like I need to properly mark for
export all symbols intended for clients of libm4.dll to use, since libm4.dll
utilizes libltdl.

Well, maybe...or not -- I think the workaround above is easier than all THAT.

Eventually, we should do something about this by offering a portable way
to decorate imports/exports for users of libtool.  Not right now.

That's a shame.  Until libtool provides helper macros that automatically
do the right thing when compiling with libtool, making M4 work on
cygwin will be a tougher process.

But libtool already does: you can use a combination of DLL_EXPORT and BUILDING_MYLIBARY to Do The Right Thing.

The real issue is that the "default" behavior that libtool expects, on cygwin/mingw -- no special -Ddefines -- is static libraries. However, on modern systems, users and developers alike universally expect shared libraries (regardless of the fact that on ELF systems, you have to pass special flags to your compiler to CREATE those shared objects: -fPIC, etc). The fact is, if I'm building foo which requires libbar, I *usually* expect that libbar is a shared library. On ELF systems, this is no big deal: I #include the headers, and link to -lbar, and it Just Works(tm) -- I'd be surprised if my foo executable didn't depend on!

The same *expectations* _should_ apply on cygwin/mingw: if foo uses libbar, it should #include libbar's headers and link -lbar, and get a foo.exe that depends on bar.dll. That should be our default assumption. If I want to link against the static libbar, I should have to explicitly do that. The linker DOES work this way: it prefers libbar.dll.a (and even bar.dll) over libbar.a -- unless you explicitly say -Bstatic. IMO, the compile process should be the same way: no special -Ddefines, you get the DLL import symbols.

But that isn't the way things work, for historical and practical reasons. And short of something really clever:
  (1) auto-import
  (2) Bruno's ideas
making DLLs "Just Work" the way I have described requires significant effort on the part of the library porter. Which is why everybody jumped onto the (1) bandwagon once it was implemented. If (2) had happened first, everybody would've jumped to it. However, as things stand (2) has to compete against entrenched inertia of (1)...

Back in the dawn of time (the glorious days of cygwin B20.1, which as everyone knows was practically perfect in every way) I ported a lot of libraries to cygwin, and "dll-ized" them. Back then, I used a magic stanza like this (there might have been an 'extern' here or there):

#ifdef __CYGWIN__
#  define LIBZ_IMPEXP
# else
#   define LIBZ_IMPEXP declspec(dllexport)
#  else
#   define LIBZ_IMPEXP declspec(dllimport)
#  endif
# endif
# define LIBZ_IMPEXP

::: libtool was not an issue, because it really only barely
::: supported DLLs at the time.  That came later, with the
::: advent of auto-import and pseudo-relocations (more below).

And then I manually decorated all the exportable symbols with LIBZ_IMPEXP (with, of course, accompanying changes in the Makefiles/configury). Obviously, these changes were massively intrusive -- BUT, in this scheme, once the library was built, clients by default would get the dllimport symbols, and since the linker would by default use the import lib, all was well. Or, a client could
when compiling and
  Wl,-Bstatic -lz -Wl,-Bdynamic
when linking, to get a static link. (Yes, it was tricky and ugly -- BUT, in this scenario, linking statically was the UNUSUAL operation. The TYPICAL dynamic link was clean).

(I actually got tired of -DTIFF_STATIC -DJPEG_STATIC -DZLIB_STATIC ... and added an extra symbol to all of the ports: -DALL_STATIC. But still...)

And then along came auto-import...and most of these porting efforts atrophied away, as they were no longer necessary in most cases (assuming you ignore Bruno's critique of auto-import's memory inefficiencies).

Then, along came pseudo-relocation support in cygwin and mingw runtimes -- which supported auto-import of multiword variables, the only missing piece above. That made the rest of the declspec()-for-cygwin ports evaporate (ignoring the problems introduced by gcc-3.3's .ro section semantics whcih don't mix well with psuedo-relocation. Strangely, it was Bruno's objection to my "fix" for this problem in gettext which lead to his really bright idea (2) above.)

And then, with these two features, libtool 1.5.x could actually be used to build DLLs on win32/cygwin fairly easily, without a lot of intrusive patches to the library code (which was SO intrusive that many upstream maintainers would not even consider accepting such patches).

The only folks still messing around with declspec() in the autotool/libtool world were (a) diehards (b) folks using mingw explicitly to build MSVC-compatible DLLs -- obviously, this only works for non-mangled ABIs: e.g. C only (c) people using GNUish shells to drive msvc/cl/lib/link tools (d) hybrids: DLLization performed first for msvc, and then re-used by the mingw build machinery added later. Very fragmented and unsystematic. No "standard" mechanism ever developed.

The closest to a "standard" was the mechanism described in the original version of the AutoBook in the section concerning w32 ports. This solution used libtool-1.4x and explicitly relied on the 1.4x-era -DDLL_EXPORT! (-DDLL_EXPORT was removed for cygwin after then introduction of auto-import, until re-introduced last June. It has always remained for mingw.)

I'm still wondering what to put in NEWS for this and the depending
change.  We should mention this somehow.  Suggestions appreciated.
"Fix regression in libltdl symbol exports on Cygwin. Side effect: LT_GLOBAL_DATA and LT_SCOPE are now explicitly defined as declspec(dllexport), bypassing auto-export logic on Cygwin. This tracks existing behavior on MinGW."

Should we also mention the side effect that you must now mark
explicit exports, since you can no longer rely on


auto-EXPORTs, actually.  The problem only occurs when:
  * creating a DLL that itself links in
  ***  a libltdl.a convenience archive
But the problem is only *detected* when linking to this poor disabled DLL.

auto-import still works -- it's just that there's nothing exported by the DLL for auto-import to use. And, there's a fairly simple workaround, that can be imposed manually as described above. It could even be automated by libltdl: if libltdl is being built as a convenience archive, define some symbol. In ltdl.h (lt_system.h in HEAD), use that symbol to force LT_SCOPE to be 'extern' even on CYGWIN|WIN32, thus overriding the DLL_EXPORT from libtool.


[***] Note that this problem should be REALLY rare: it only happens when
(1) you have a set of library code (call it 'A') that includes necessary support for building as a dll (2) BUT you're actually using it as a convenience library when building a DIFFERENT dll (call it 'B') (3) AND "A" and "B" each follow different standards for symbol decoration (e.g. libm4 dll has no symbol decoration whatsover; but ltdl follows a -DDLL_EXPORT-driven scheme)

You do NOT run into this problem [you get a different issue!] when 'A' is built as an external, static library. In that case, libtool will refuse to create 'B' with "static" dependency "A", thanks to its justified (on ELF) paranoia about mixing "PIC" and "non-PIC" code. Yes, yes, there's no real difference on win32 -- but cygwin-libtool still honors this paranoia (as does mingw-libtool, I think).

reply via email to

[Prev in Thread] Current Thread [Next in Thread]