Re: exception propagated across module boundary, type unknown

[ben mitch]

Paul Pluzhnikov wrote:
> First confusion point: the "manager" is not in fact an executable.
> It is a shared library just like any other.

yes sorry should have made that clear.

> > the
> > exception is now caught correctly. if i restart matlab, the exception
> > is uncaught on the first run after restart.
>
> I can't think of any explanation for this ...

darn it, nor can i. seems my speculations about late binding just 
weren't accurate then.

> >  > Is there any difference between LD_DEBUG=... output on first
> >  > (failing) and second (successful) run?
> >
> > i had thought this was not possible to do originally since i am
> > calling the executable from matlab. i have tried to set the LD_DEBUG
> > var from within matlab
>
> That would be too late: LD_DEBUG is examined only when the process
> starts. Once MatLab is running, any settings of LD_DEBUG will only
> affect its children, but not MatLab itself.

right. i thought i would be looking for link steps that took place when 
the manager shared library was invoked, and so we move on...

>>bizarrely, no linking appears to be performed
> > throughout the time bracketed by the first invocation of the manager
> > and return from the second invocation.
>
> All "interesting" linking will happen *before* you see any output
> from the first invocation of manager.

sorry, used a semantic shortcut there that obfuscated what i actually 
did... i did indeed use 'print hello' or its equivalent to delineate 
when linking occurred. in particular, i performed:

export LD_DEBUG=...
invoke matlab > file 2>&1

file:
========================================
(lots of output from ld)
                                        :A
(matlab displays its startup message)
(my script is automatically and silently invoked)
                                        :B
(now we are in my script)
print 'op1'
call manager
print 'op2'
call manager
print 'end'
                                        :C
========================================

and between B and C, no output from ld appears. i might argue that 
matlab pre-compiles the script, and loads any shared libraries it needs 
before actually running the script (hence before B), except that no 
output from ld actually appears anywhere between A and C - and, as i 
mentioned, the shared libraries are actually loaded and resolved (or at 
least ld gives related output) _before_ A, i.e. before matlab has even 
loaded. i can't think it's reasonable to speculate that matlab pre-loads 
the script that it knows it will execute once it finishes loading _and_ 
pre-loads any libraries that script might need?! that just doesn't seem 
like an intuitive way to arrange things...

as to what the actual output of ld looks like, i give a quick example, 
the first reference to the custom exception type EXCEPTION in the dump...

============================================

     10825:	symbol=_ZTS9EXCEPTION;  lookup in 
file=/usr/share/matlab704/bin/glnx86/MATLAB
     10825:	symbol=_ZTS9EXCEPTION;  lookup in 
file=/usr/share/matlab704/bin/glnx86/libut.so

<snip>

     10825:	symbol=_ZTS9EXCEPTION;  lookup in 
file=/usr/share/matlab704/sys/java/jre/glnx86/jre1.5.0/lib/i386/libverify.so
     10825:     symbol=_ZTS9EXCEPTION;  lookup in file=/lib/libnsl.so.1
     10825:	symbol=_ZTS9EXCEPTION;  lookup in 
file=/home/mitch/BRAHMS/glnx86/bin/libbrahms-core.so
     10825:	binding file 
/home/mitch/BRAHMS/glnx86/bin/components/libbrahms-data-listSpikes-0.so 
to /home/mitch/BRAHMS/glnx86/bin/libbrahms-core.so: normal symbol 
`_ZTS9EXCEPTION'

============================================

and following appearances as far as i have checked look similar (i.e. 
bind successfully) - is there a keyword to search for in case of error? 
i'm guessing i would have got some sort of loading error reported to me 
if binding failed in any case. (NOTE: sorry this is actually being built 
at ~/BRAHMS/glnx86/bin not ~/bin as i have been indicating, i just 
shortened it for readability).

> > previous to matlab even
> > starting, functions within the manager/core/process application appear
> > to be resolved - does this mean linux caches the libraries that an
> > application uses, and pre-resolves them?
>
> No. You probably just didn't determine correctly where the "MatLab
> is loaded and now starts running the user script" boundary is.

this may still be true since matlab could be doing stuff in any order it 
chooses, but superficially it doesn't look like it.

> > Meanwhile, this just in... after performing two runs in matlab, so the
> > exception is now being recognised correctly, the following
> > actions/effects are noted:
> >
> > * rebuild [manager] : next run, exception recognised correctly
> > * rebuild [manager-parts], [manager] : next run, exception recognised
> > * correctly
> > * rebuild [process] : next run, exception recognised correctly
> > * rebuild [core] : next run, exception recognised correctly
> > * rebuild [everything] : next run, exception recognised correctly
>
> This is because MatLab doesn't reload "manager", "process" or "core":
> the old copies are still in-core and are being used.

i'm not sure that's true - if i rebuild (say) process after making a 
change to the code, the change to the code is reflected in a subsequent 
run. same goes for any modification of the manager (and the core i 
think...? i can check).

that's just made me realize i may have misled a bit, beg your pardon... 
note that the linking between manager and core and between process and 
core is at compile&link time (see compiler lines in previous post), 
whereas the linking between manager and process (manager calls 
CreateObject() in process) is performed using dlopen/dlsym (i.e. the 
process is not available when the manager is built). i don't know if 
this is relevant, sorry if it changes everything... :|

> > * restart matlab : next run, not recognised
>
> Now you are re-loading the shared libraries, and they fail the
> same way. This is self-consistent (and consistent with my mental
> model of what's happening).

my mental model is still deficient i'm afraid :)

so where are we at? all linkages are being correctly resolved 
(eventually) and all linkage reported by ld takes place before point A 
in the above pseudo-code. therefore (i think) all linkages are correctly 
resolved before point A. so that's not the problem, it's something else. 
right? errr... errr...

cheers