[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-apl] using c libs in apl?

From: Elias Mårtenson
Subject: Re: [Bug-apl] using c libs in apl?
Date: Thu, 9 Feb 2017 17:05:31 +0800

I wasn't referring to the management of APL memory, but rather native memory used when calling functions through the FFI.

As an example, I've been recently working on integrating GSSAPI in Emacs (I've previously integrated the same library in Common Lisp), and as an example, let's take a look at a typical C function call in GSSAPI:

A simple function is gss_display_name() which is used to retrieve the name of a principal as a string, given the principal object (returned from a previous function call). Here is the signature:

OM_uint32 gss_display_name(
    OM_uint32       *minor_status,
    name_t           input_name,
    gss_buffer_desc *output_name_buffer,
    gss_OID         *output_name_type);

Here's how you use the function, assuming the name is in the variable ‘name’:

    gss_buffer_desc out;
    gss_OID out_type;

    int minor;
    int major = gss_display_name(&minor, name, &out, &out_type);
    if(GSS_ERROR(major)) {
        // there was an error, and the details about the error can be found in major and minor

    // The name is now available in a string located at out.value with the length out.length
    // Making this extra complicated is that out.value is not nul-terminated.
    char *name_as_string = malloc(out.length + 1);
    strncpy(name_as_string, out.value, out.length);
    name_as_string[out.length] = 0;

    // We now have the name as a string in the variable name_as_string.
    // There are some other API calls needed to release the memory allocated by the call to gss_display_name
    // but I'm ignoring that for the purpose of the example.

All right, with this in mind, we'll have to figure out an APL API that allows me to do this, and even more complex stuff. It's possible, but not easy. Here's an attempt at doing so that I'm just typing out as I see it just to have something discuss around:

  ⍝ The size of a gss_buffer_desc consists of 2 pointers,
  ⍝ which makes it 16 bytes on 64-bit platforms and 8 bytes
  ⍝ on 32-bit platforms.
  gss_buffer_desc_size ← 16
  out ← ⎕FFI_Alloc gss_buffer_desc_size

  ⍝ The gss_OID type is just a pointer, so 8 bytes on 64-bit platforms
  gss_OID_size ← 8
  out_type ← ⎕FFI_Alloc gss_OID_size

  minor ← ⎕FFI_Alloc 4    ⍝ 32-bit number

  ⍝ We need to specify the datatype of the return value, so we'll use an
  ⍝ axis argument for that.
  major ← 'gss_display_name' ⎕FFI_Call minor[Type_Int32] name out out_type

  ⍝ The C macro GSS_ERROR expands to some bit-fiddling,
  ⍝ but it's nothing we can't deal with in APL. There is
  ⍝ an error if any of the most-significant 16 bits are set.
  is_error ← 0 ≠ +/((32⍴2)⊤4294901760) ∧ (32⍴2)⊤major

  ⍝ Extract a pointer from 8 bytes after the top of the struct that out points to
  out_value ← 8 ⎕FFI_Dereference_Pointer out

  ⍝ Extract a 64-bit number from the top of the struct
  out_length ← 0 ⎕FFI_Dereference_Int64 out

  ⍝ Construct an APL string from an array of UTF-8 characters.
  ⍝ The idea here is that the left argument specifies the number of bytes to
  ⍝ copy, and if the function is called monadically it will simply copy
  ⍝ until a terminating NUL byte.
  name_as_string ← out_length ⎕FFI_MakeString out_value

  ⍝ Finally, free the memory we allocated previously
  ⎕FFI_Free out
  ⎕FFI_Free out_type

I don't think this can be made much simpler, and this is a reasonably simple real-world example of C API's that one needs to call. I've adopted this example from my Common Lisp code, and if you want to look at how it's done there you're welcome to look at that code: https://github.com/lokedhs/cl-gss/blob/master/src/cl-gss.lisp#L136


On 9 February 2017 at 00:50, Juergen Sauermann <address@hidden> wrote:
Hi Elias,

the latest libapl API (libapl.h) may give some ideas. It uses a 2-step approach like GNU APL internally: first
create a value with a given shape/rank and then set the elements of its ravel. The value must be released
explicitly when no longer needed. This is because libapl is a C interface not a C++ interface. Therefore the
Value_P magic cannot be used in a C library. In C++ things are much simpler because you could use
Value_P objects, which release the underlying APL value automatically.

/// Jürgen

On 02/08/2017 05:33 PM, Elias Mårtenson wrote:
This is something I might want to take a look at. I think the most difficult part of implementing this is to decide on a nice way to map the libffi API to APL in a natural way.

I'm thinking of providing a quad-function that allows you to declare a C function and their arguments (and associated types). That way you don't have to mess with datatypes when it comes to actually calling the native functions.

Still, you need to have constructs that allows you to allocate memory, as well as functions to access the content of said memory. I have no idea how such an API should look in APL.

I might take a look at this, but right now I'm working on some other projects so I don't have time. 


On 8 Feb 2017 23:05, "Juergen Sauermann" <address@hidden> wrote:

I had a quick look at both the C code from the www.jsoftware.com <http://www.jsoftware.com/help/user/call_procedure.htm> and fromhttps://github.com/libffi/libffi <https://github.com/libffi/libffi>
My first impression is that the former is quite hack-ish.

But I haven't worked with libffi myself, so I cant really say if it lives up to its promises.
If it does then my vote would definitely be for *libffi*.

Another plus for *libffi* is that it is available as debian package.

/// Jürgen

On 02/08/2017 01:38 AM, Elias Mårtenson wrote:
This would be really neat to have, but as someone who has written a lot of FFI (foreign function interface) code in Common Lisp which has a very powerful such interface, there are a lot of nuances that needs to be covered in order to have a decent FFI.

For example, what if you need to call a function which accepts a struct as its first argument which contains a pointer to another struct which in turn has a list of unsigned integers of size Foo (defined with a typedef in a .h file of course). The second argument being a pointer to a callback function.

That just gives a small idea of the issues one would come across.

Thankfully there is a C library, libffi, which can help here. It's designed to assist when creating an FFI for something like GNU APL. I recommend anyone who considers taking up this project to investigate it.

libffi can be found here: https://github.com/libffi/libffi

I certainly would really like it if this was implemented.


On 8 Feb 2017 03:01, "Juergen Sauermann" <address@hidden <mailto:address@hiddenline.de>> wrote:

    Hi Xiao-Yong,

    I believe this could be achieved by a single "master"-native
    function which then loads the
    actual DLL as specified by the arguments of the master function.
    Referring to the example in
    link you gave below:

    *a=: 'kernel32 GetProfileStringA s *c *c *c *c s' b=:
    'windows';'device'; 'default'; (32$'z');32 a cd b
    |31|windows|device|default|HP LaserJet 4P/4MP,HPPCL5MS,LPT |32|

    This would become in GNU APL:

    *a← 'kernel32 GetProfileStringA s *c *c *c *c s' b← 'windows'
    'device' 'default' (32⍴'z') 32******'universal-dll-loader' ⎕FX 'cd'****** a cd b ⍝ dlopen("kernel32.dll"), dlsym("GetProfileStringA") on
    first access,****⍝ and call GetProfileStringA with argument b*

    The *universal-dll-loader.so* needs to be written only once and
    contains mainly the code from github below. That code somehow goes
    into the *native/template_F12.cc* code of GNU APL and thats it. If
    you need help doing this then please let me know. /// Jürgen

    On 02/07/2017 06:30 PM, Xiao-Yong Jin wrote:
    It would be nice if one doesn't need to write wrappers and the APL system can do the
    structure conversions within the APL interpreter.  In J, you can dlopen a library
    and pass J values directly without writing and compiling C, see


    and the relevant code is at


    It would simplify using external libraries a lot.

    On Feb 4, 2017, at 7:38 AM, Juergen Sauermann<address@hiddenonline.de>
    <mailto:address@hiddenline.de>  wrote:


    yes there is: native functions. You can load shared libraries and ⎕FX functions in
    them to be called from APL code. The src/native directory contains a few templates
    that you can use as a starting point and to call your favourite library from them.

    Of course you need to provide wrappers from/to APL values to/from the data
    structures expected or produced by the libraries.

    Coming back to your other problems, if you do not like the terminal I/O of GNU APL, then
    you can write your own one and call libapl from it. I have extended libapl recently, giving
    you the full functionality of GNU APL without the specific ways how it handles terminal IO.

    /// Jürgen

    On 02/04/2017 02:52 AM,address@hidden <mailto:address@hidden>  wrote:
    is there method for loading a c lib and using it in apl ?  cdecl?   like this in fpc?


reply via email to

[Prev in Thread] Current Thread [Next in Thread]