Re: [Bug-apl] Bug: GNU APL does not call start_input after a )LOAD

[Elias Mårtenson]

First of all, sorry about the benchmark numbers. I thought I already sent them but I see now that I had forgotten about it. I'll do it tomorrow.

About the libraries, I see what you are saying. Having a library ID is definitely one way of dealing with this (I suppose the package manager can be used to pull it in if it's missing on the system).

However, the Emacs mode is different. A non-emacs user would not want to (and shouldn't) have the library loaded. Likewise, if I load a dump from someone who is not using Emacs, I don't want to lose the Emacs mode that I already loaded.

I think that the Emacs mode native library and others, such as the SQL one, are fundamentally different in the sense that the former attaches to the runtime session and never interacts with the programs the user writes, while the other is a "normal" library used by a specific program that only differs from a normal APL program by the language the library is written in.

The library ID system is definitely a good thing for the latter type of native library.

However, I believe a separate model is needed for the former type. In fact, the Emacs mode library doesn't even need a function entry point. I can imagine other libraries of the same type, though none as obvious as the editor integration stuff.

The former type need to load, stay in a single session, and never interact with the user in any visible way. This includes being included in a dump file. Even seeing the native entry point in the output of )FNS can be confusing, and I would really like to be able to avoid that one.

Regards,

Elias

On 10 May 2014 22:14, Juergen Sauermann <address@hidden> wrote:

Hi Elias,

I am not sure if the observed behavior is actually wrong. If you load two libraries with different paths
then you get two different libraries and each is initialized. Right now NativeFunction checks if a shared
library with the same path (i.e. directories and filename) exists and does not load it twice if so.

Your expectation seems to be that a library with the same name but stored in different places should
also not be loaded twice. A consequence of this would be that you cannot replace a library with a different
version and the same name. In your case it would mean that if you start in emacs mode then you cannot
load or replace the emacs mode library in the same interpreter session. This may suit you in relation to
the current problem but could fire back in other cases.

What might be somewhat cleaner is to have a function get_library_id() that returns some identification string,
and to stop the loading if a duplicate id is detected (or let the library decide how it would like to proceed).

BTW. I would still be interested in the result files of your 80-core benchmark.

/// Jürgen

On 05/10/2014 03:09 PM, Elias Mårtenson wrote:

The problem is not related to the unloading of functions, but that the call to )LOAD triggers a reload of the function, even though the old one is still active.

That reload is what corrupts the Emacs mode in two different ways, either of which would cause the Emacs mode to hang:

• The reload triggers another call to the initialisation code (called by get_signature). This causes an already initialised library to reinitialise. I could deal with this using a flag, but…
• The library that the )LOAD loads is not the same as the one that was originally loaded. In my case, the Emacs native library was original loaded form /home/elias/prog/gnu-apl-mode/native/libemacs.so, but the call to )COPY then loads /home/elias/src/apl/dist/lib/apl/libemacs.so (that's where my development version of GNU APL is installed). So yes, the library is now loaded twice, causing no end of trouble.

Regards,

Elias

On 10 May 2014 21:02, Juergen Sauermann <address@hidden> wrote:

Hi Elias,

I believe this relates to an earlier discussion about removing of callbacks.

Currently a library can decide how it wants to be handled  by )LOAD (actually by )ERASE
which is triggered by )LOAD):

A. be dlcosed() (and then needs to clear/restore its callbacks) or
B. remain loaded.

The default is B as requested earlier (so that the callbacks remain intact). If you
want A. instead then define a function close_fun() and make it return true when called.
Define means that get_function_mux() returns a pointer to it.

Now, if you go for A. then you should restore your callbacks in close_fun().
On the other hand, if you go for B. then you could be called twice and should prepare for that
(a reference counter ++'ed in get_sig() and --'ed in close_fun()). Dito for the same
library used under different names in the same WS, so namespaces would not help).

dlopen() clains to take care of not being loaded twice, but the details are not very clear, eg.
symbolic links to libs, copy of libs etc.

/// Jürgen

On 05/08/2014 03:28 PM, Elias Mårtenson wrote:

OK, thanks for the hints. I've spent some time looking at this issue and I now have an idea what is happening.

When you issue )LOAD, the native library becomes loaded again, even though it's already loaded. There are two issues with this:

First and foremost, after the )LOAD the version of libemacs that is shipped with GNU APL was loaded, not the one correct onw (which happens to be in my development directory). If you intend to call dlopen on the same library, it's important that it's actually the same library and not another one with the same name in a different directory.

Secondly, after loading this library again, GNU APL calls the get_sig() function again. Even if the dlopen() call had returned the same pointer, this would be problematic because the library performs some initialisation stuff in this function call (as you previously had recommended when I asked where I should run the initialisation). Trying to reinitialise an already initialised library is not good.

In short, I think that the idea of using a normal function as entry point into the native code is problematic. I wouldn't mind a redesign of this so that native code lives in a namespace separate from the normal code so it's unaffected by calls to )CLEAR, )ERASE etc.

Regards,

Elias

On 8 May 2014 18:51, Juergen Sauermann <address@hidden> wrote:

Hi,

could you please print the value of start_input  just before line 223 in Input.cc like:

Input::get_user_line(const UCS_string * prompt)
{
Q(start_input)
   if (start_input)   (*start_input)();
...

On my machine it looks OK (even if I don't set it):


     This program is free software, and you are welcome to redistribute it
         according to the GNU Public License (GPL) version 3 or later.
                                      

start_input:         '0' at Input.cc:223
      )load DIJKSTRA
SAVED 2014-5-8  10:42:39 (GMT+2)
start_input:         '0' at Input.cc:223

/// Jürgen

On 05/08/2014 05:07 AM, Elias Mårtenson wrote:

This was reported in this thread: https://github.com/lokedhs/gnu-apl-mode/issues/7

This problem seems to be caused by a bug in GNU APL. When the user calls )LOAD to load a workspace, the start_input callback function is not called before control is returned back to the user.

(because of this, the Emacs mode just sits there, hung, waiting for a call to start_input that never happens).

Regards,

Elias