Re: [PATCH] [cygwin|mingw] fix dlpreopen with --disable-static

[Charles Wilson]

Ralf Wildenhues wrote:
> Hello Charles,
> 
> thanks for the patch!  Quoting a bit out of order:
> 
> * Charles Wilson wrote on Sat, May 31, 2008 at 07:01:45PM CEST:
>> * libltdl/config/ltmain.m4sh (func_enable_tag): allow
>> --verbose to set opt_verbose
> [...]
>> A) libtool --verbose does not actually set opt_verbose. In fact,
>>    nothing ever sets opt_verbose true.  Should all uses of opt_verbose
>>    be replaced by !opt_silent, or should (as I have done in this patch)
>>    --verbose set both opt_silent false, and opt_verbose true?
> 
> Well, --verbose is documented to be a reversal of --silent, and
> documented to be the default.  The fact that opt_verbose is never set is
> a limitation.  If fixed, that should better happen in a separate patch.

Well, if --verbose is really the negation of --silent, then (unless the
functionality is extended as you describe) the effect of --verbose
should be to only unset opt_silent (which it does), and there should not
exist any 'opt_verbose' variable.

> Several possibilities:
> - a new switch for being very verbose.
> - allowing
>      --verbose
>      --silent --verbose --verbose
> 
> to output verbose output.  That way one can still undo the effects of
> --silent without verbose output; however, now the user needs to know
> whether --silent was passed earlier on the command line (this is a
> backward incompatibility).  Not nice (imagine --silent being passed in
> AM_LIBTOOLFLAGS).

In this case, to avoid backwards compatibility issues, the new "I want
really talkative output, but not --debug" option should be something
other than '--verbose' -- because that already has a specific meaning:
negate--silent.

--chatty?  (I know, it's idiomatic and that's bad. It's more a
tongue-in-cheek suggestion anyway).

Regardless, I think the current (2? 3?) usages of opt_verbose should be
changed to !opt_silent.

>> B) func_win32_libid() gives some confusing errors to users
>>    when (a) using recursive make, and (b) MAKEFLAGS does not
>>    contain $OBJDUMP. Add a diagnostic error message, rather
>>    than allowing $SED to die a horrible death.
> 
> Why should external variables ever matter here?  ,,,
> Anyway, if this should be needed, then it should be a separate change as
> well ...

Actually, this may no longer be necessary given the _LT_DECL_OBJDUMP
changes (I /said/ this was an old patch).  Here's the thread:
http://cygwin.com/ml/cygwin/2008-09/msg00552.html

> 
> Now to the meat of the patch:
> 
>> When configuring with --disable-static, dlpreopen is very confused.
>> First, libtool tries to extract the symbols -- using $NM and
>> $global_symbols_pipe -- from the DLL. That works...poorly:
>>
>> Here's a snippet from helldl.exeS.c when --disable-static:
>> ======
>> /* External symbol declarations for the compiler. */
>> extern int _CTOR_LIST__();
>> extern int _DTOR_LIST__();
>> extern char _RUNTIME_PSEUDO_RELOC_LIST_END__;
>> extern char _RUNTIME_PSEUDO_RELOC_LIST__;
>> extern int __CTOR_LIST__();
>> extern int __DTOR_LIST__();
>> ...
>> extern int printf();
>> extern int puts();
>> extern int realloc();
> 
> OK, I see that this is problematic.  What I don't understand yet is:
> is there a way to extract only the interesting symbols from the DLL?
> I don't yet understand why we have to move to the import library.

Because it's extremely tricky. You have to use objdump (not nm), and
then search for the exported symbols which is non-trivial because you
need a stateful parser -- maybe an awk program...

objdump -p cyghello-2.dll | awk '/^Export Address Table/{foundEdata=1}
/[Ordinal/Name Pointer] Table/{foundTable=1}
{if (foundTable==1 && foundEdata==1) print $0 }'

is a start. But, the objdump output doesn't indicate whether a symbol is
a function or data. To figure that out (from objdump) you need to parse
the reloc section of the report...or cross-reference to the nm output,
which indicates function or data, but doesn't tell you which DLL the
symbol actually came from: $arg, or one of $arg's dependencies.

The reason nm is not acceptable, is it detects all symbols imported and
exported by the DLL (assuming the DLL has not had its symbol information
stripped, in which case nm reports nothing).  How can we construct a
regex that accepts _foo, but rejects _free?

cyghello-2.dll:6c641020 T _foo
cyghello-2.dll:6c6413c0 T _free

The ugliest part of my patch is the fallback code for deducing the
dllname from an import library.  But that's *pretty* compared to mucking
with objdump output.

Alternatively, libtool used to embed an "impgen" program which could be
ressurected to generate the symbol lists we need (rather than a .def
file, as it used to do).  A better solution would be to push that
functionality into dlltool ("Hmm. I need to generate a symbol list or
def file for a DLL." but 'dlltool --output-def=my.def my.dll' doesn't do
what you expect).  Even so, the core functionality of impgen would need
some improvements (especially so as to indicate DATA items, which it
doesn't do at present).
http://loreley.ath.cx/cygwin/impgen/impgen.c

But then, you're back to requiring a very recent binutils, or providing
a fallback -- see "objdump ugliness" above.

> Once I understand that, I can better judge the rest of the patch.

Well, one reason I sat on this for so long was the 'fallback' mechanism
for deducing the dll name from the import library was just so...hideous.
And it wasn't a fallback -- it was the only mechanism since I hadn't yet
enhanced dlltool.  The only reason to allow it is because (hopefully)
that ugly fallback code can get flagged with a warning, and maybe in a
year or so get removed.

Well, that, and it fixes a test that currently fails.

> A couple of general remarks though:
> - there is no need to use $EGREP with literals; grep can do that, or
>    case `$command` in *literal*) ... ;; esac
>   which avoids a second fork.
> - the code uses eval several times; I think it is really only necessary
>   in lines that contain, for example, '| $global_symbol_pipe ...', or
>   $file_magic_cmd.

Ack.

> for example mentioning in the mail
> whether you considered cygwin or cegcc or so would be helpful for
> review).

cygwin and mingw yes. I know nothing about cegcc.

--
Chuck