Re: [Axiom-developer] New dynamic library access in GCL

[Gernot Hueber]

Dear Camm,

this is great! Thanks for your effort related to this issue.

First I want to ask, is this version running on FreeBSD as well and how
can I access the source for this version?

Secondly, what types are supported by defdlfun and does it make sense to
finish cffi-gcl from your point. cffi seems to be the most actively
developed and used ffi for lisp and opens the door to use swig as
well :-). There is already some work done to support GCL, but still more
effort needed.

Best regards,

Gernot

On Fri, 2007-06-15 at 20:15 -0400, Camm Maguire wrote:
> Greetings!  GCL now has the ability to access arbitrary external
> shared library routines in a persitent fashion -- i.e. the binding is
> kept across image saves:
> 
> =============================================================================
> address@hidden:/fix/t1/camm/debian/gcl/tmp/tmp/foo1$ unixport/saved_gcl
> GCL (GNU Common Lisp)  2.7.0 CLtL1    Jun 15 2007 19:45:46
> Source License: LGPL(gcl,gmp,pargcl), GPL(unexec,bfd,xgcl)
> Binary License:  GPL due to GPL'ed components: (XGCL READLINE BFD UNEXEC)
> Modifications of this banner must retain notice of a compatible license
> Dedicated to the memory of W. Schelter
> 
> Use (help) to get some basic information on how to use GCL.
> 
> Temporary directory for compiler files set to /tmp/
> 
> >(|libm|:|erf| 1.0)
> 
> Error: ERROR "Cannot find the external symbol erf in #<\"libm\" package>."
> Fast links are on: do (si::use-fast-links nil) for debugging
> Signalled by READ.
> ERROR "Cannot find the external symbol erf in #<\"libm\" package>."
> 
> Broken at READ.  Type :H for Help.
> >:q
> 
> >(si::show-lib-syms)
> 
> (LIB:|libm| 1074447640 #<"libm" package>) 
> (|libm|:|atan| 1094141472 #<compiled-function |libm|:|atan|>) 
> (|libm|:|ctan| 1094156208 #<compiled-function |libm|:|ctan|>) 
> (|libm|:|csqrt| 1094159152 #<compiled-function |libm|:|csqrt|>) 
> (|libm|:|clogf| 1094183232 #<compiled-function |libm|:|clogf|>) 
> (|libm|:|acosh| 1094146080 #<compiled-function |libm|:|acosh|>) 
> (|libm|:|ccosh| 1094153584 #<compiled-function |libm|:|ccosh|>) 
> (|libm|:|expf| 1094176432 #<compiled-function |libm|:|expf|>) 
> (|libm|:|atanhf| 1094176064 #<compiled-function |libm|:|atanhf|>) 
> (|libm|:|casin| 1094154864 #<compiled-function |libm|:|casin|>) 
> (|libm|:|cexpf| 1094181648 #<compiled-function |libm|:|cexpf|>) 
> (|libm|:|acosf| 1094175504 #<compiled-function |libm|:|acosf|>) 
> (|libm|:|sqrtf| 1094180496 #<compiled-function |libm|:|sqrtf|>) 
> (|libm|:|exp| 1094146864 #<compiled-function |libm|:|exp|>) 
> (|libm|:|atanh| 1094146512 #<compiled-function |libm|:|atanh|>) 
> (|libm|:|ccosf| 1094184128 #<compiled-function |libm|:|ccosf|>) 
> (|libm|:|ctanh| 1094156960 #<compiled-function |libm|:|ctanh|>) 
> (|libm|:|cosh| 1094146672 #<compiled-function |libm|:|cosh|>) 
> (|libm|:|ccoshf| 1094182688 #<compiled-function |libm|:|ccoshf|>) 
> (|libm|:|cosf| 1094172320 #<compiled-function |libm|:|cosf|>) 
> (|libm|:|atanf| 1094172032 #<compiled-function |libm|:|atanf|>) 
> (|libm|:|cos| 1094141792 #<compiled-function |libm|:|cos|>) 
> (|libm|:|cacos| 1094157584 #<compiled-function |libm|:|cacos|>) 
> (|libm|:|tanh| 1094145584 #<compiled-function |libm|:|tanh|>) 
> (|libm|:|ctanf| 1094185024 #<compiled-function |libm|:|ctanf|>) 
> (|libm|:|csinhf| 1094182144 #<compiled-function |libm|:|csinhf|>) 
> (|libm|:|tanf| 1094175168 #<compiled-function |libm|:|tanf|>) 
> (|libm|:|tan| 1094145536 #<compiled-function |libm|:|tan|>) 
> (|libm|:|asin| 1094146208 #<compiled-function |libm|:|asin|>) 
> (|libm|:|sinh| 1094150832 #<compiled-function |libm|:|sinh|>) 
> (|libm|:|csin| 1094155520 #<compiled-function |libm|:|csin|>) 
> (|libm|:|sinf| 1094175120 #<compiled-function |libm|:|sinf|>) 
> (|libm|:|cabs| 1094152304 #<compiled-function |libm|:|cabs|>) 
> (|libm|:|sin| 1094145488 #<compiled-function |libm|:|sin|>) 
> (|libm|:|catanhf| 1094187072 #<compiled-function |libm|:|catanhf|>) 
> (|libm|:|coshf| 1094176224 #<compiled-function |libm|:|coshf|>) 
> (|libm|:|catanh| 1094158672 #<compiled-function |libm|:|catanh|>) 
> (|libm|:|fabs| 1094144096 #<compiled-function |libm|:|fabs|>) 
> (|libm|:|catanf| 1094183504 #<compiled-function |libm|:|catanf|>) 
> (|libm|:|tanhf| 1094175216 #<compiled-function |libm|:|tanhf|>) 
> (|libm|:|acoshf| 1094175632 #<compiled-function |libm|:|acoshf|>) 
> (|libm|:|asinh| 1094141232 #<compiled-function |libm|:|asinh|>) 
> (|libm|:|csinh| 1094152928 #<compiled-function |libm|:|csinh|>) 
> (|libm|:|asinhf| 1094171792 #<compiled-function |libm|:|asinhf|>) 
> (|libm|:|atan2f| 1094175888 #<compiled-function |libm|:|atan2f|>) 
> (|libm|:|asinf| 1094175760 #<compiled-function |libm|:|asinf|>) 
> (|libm|:|sinhf| 1094180368 #<compiled-function |libm|:|sinhf|>) 
> (|libm|:|atan2| 1094146336 #<compiled-function |libm|:|atan2|>) 
> (|libm|:|csinf| 1094184448 #<compiled-function |libm|:|csinf|>) 
> (|libm|:|cabsf| 1094181552 #<compiled-function |libm|:|cabsf|>) 
> (|libm|:|fabsf| 1094174288 #<compiled-function |libm|:|fabsf|>) 
> (|libm|:|casinhf| 1094186160 #<compiled-function |libm|:|casinhf|>) 
> (|libm|:|logf| 1094179104 #<compiled-function |libm|:|logf|>) 
> (|libm|:|casinh| 1094157680 #<compiled-function |libm|:|casinh|>) 
> (|libm|:|clog| 1094154112 #<compiled-function |libm|:|clog|>) 
> (|libm|:|casinf| 1094183888 #<compiled-function |libm|:|casinf|>) 
> (|libm|:|ctanhf| 1094185616 #<compiled-function |libm|:|ctanhf|>) 
> (|libm|:|csqrtf| 1094187472 #<compiled-function |libm|:|csqrtf|>) 
> (|libm|:|log| 1094149536 #<compiled-function |libm|:|log|>) 
> (|libm|:|cacoshf| 1094186624 #<compiled-function |libm|:|cacoshf|>) 
> (|libm|:|cacosh| 1094158144 #<compiled-function |libm|:|cacosh|>) 
> (|libm|:|catan| 1094154432 #<compiled-function |libm|:|catan|>) 
> (|libm|:|cacosf| 1094186080 #<compiled-function |libm|:|cacosf|>) 
> (|libm|:|cexp| 1094152400 #<compiled-function |libm|:|cexp|>) 
> (|libm|:|acos| 1094145952 #<compiled-function |libm|:|acos|>) 
> (|libm|:|sqrt| 1094150976 #<compiled-function |libm|:|sqrt|>) 
> (|libm|:|ccos| 1094155168 #<compiled-function |libm|:|ccos|>) 
> (|libm|:|abs| 1090798240 #<compiled-function |libm|:|abs|>) 
> (LIB:|libc| 1074448272 #<"libc" package>) 
> (|libc|:|setjmp| 1090786864 NIL) 
> (|libc|:|feof| 1090993296 NIL) 
> (|libc|:|memset| 1091054096 NIL) 
> (|libc|:|getc| 1090994720 NIL) 
> (|libc|:|bzero| 1091054432 NIL) 
> (|libc|:|putc| 1090995664 NIL) 
> 
> >(in-package 'compiler)
> 
> #<"COMPILER" package>
> 
> COMPILER>(defdlfun (:double "erf" "libm.so") :double)
> 
> |libm|:|erf|
> 
> COMPILER>(compile *)
> 
> ;; Compiling /tmp/gazonk_18613_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> ;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
> (Debug quality ignored)
> ;; Finished compiling /tmp/gazonk_18613_0.o.
> ;; Loading /tmp/gazonk_18613_0.o
>  ;; start address -T 0xa02000 ;; Finished loading /tmp/gazonk_18613_0.o
> #<compiled-function |libm|:|erf|>
> NIL
> NIL
> 
> COMPILER>(|libm|:|erf| 1.0)
> 
> 0.84270079294971489
> 
> COMPILER>(|libm|:|erf| 1.0s0)
> 
> Correctable error: TYPE-ERROR :DATUM 1.0S0 :EXPECTED-TYPE LONG-FLOAT NIL
> Fast links are on: do (si::use-fast-links nil) for debugging
> Signalled by EVAL.
> If continued: choose a new value
> TYPE-ERROR :DATUM 1.0S0 :EXPECTED-TYPE LONG-FLOAT NIL
> 
> Broken at EVAL.  Type :H for Help.
> COMPILER>>:q
> 
> Top level.
> COMPILER>(disassemble '(lambda (x) (|libm|:|erf| x)) nil)
> 
> ;; Compiling /tmp/gazonk_18613_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> ;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
> (Debug quality ignored)
> ;; Finished compiling /tmp/gazonk_18613_0.o.
> 
> #include "gazonk_18613_0.h"
> void init_code(){do_init((void *)VV);}
> /*    local entry for function CMP-ANON       */
> 
> static object LI1(V2)
> 
> object V2;
> {      VMB1 VMS1 VMV1
>       goto TTL;
> TTL:;
>       {object V3 = (/* erf */(*LnkLI0)((V2)));VMR1
>       (V3);}
>       return Cnil;
> }
> static object  LnkTLI0(object first,...){object V1;va_list 
> ap;va_start(ap,first);V1=(object )call_proc_new(((object)VV[0]),0,0,(void 
> **)(void *)&LnkLI0,1,first,ap);va_end(ap);return V1;} /* erf */
> #(#(erf
>     (%INIT
>      . #((LET ((*DISABLE-RECOMPILE* T))
>            (MFSFUN 'CMP-ANON 0 1 0)
>            (ADD-HASH 'CMP-ANON '((T) T) '((erf (FLOAT) T))COMPILER
> libmerf-
>                '/tmp/gazonk_18613_0.lsp))
>          (DO-RECOMPILE)))))
> static object LI1();
> #define VMB1
> #define VMS1
> #define VMV1
> #define VMR1(VMT1) return(VMT1);
> #define VM1 0
> static void * VVi[2]={
> #define Cdata VV[1]
> (void *)(LI1)
> };
> #define VV (VVi)
> static object  LnkTLI0(object,...);
> static object  (*LnkLI0)() = (object (*)()) LnkTLI0;
> NIL
> 
> COMPILER>(disassemble '(lambda (x) (declare (long-float x)) (|libm|:|erf| x)) 
> nil)
> 
> ;; Compiling /tmp/gazonk_18613_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> ;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
> (Debug quality ignored)
> ;; Finished compiling /tmp/gazonk_18613_0.o.
> 
> #include "gazonk_18613_0.h"
> void init_code(){do_init((void *)VV);}
> /*    local entry for function CMP-ANON       */
> 
> static double LI1(V2)
> 
> double V2;
> {      VMB1 VMS1 VMV1
>       goto TTL;
> TTL:;
>       {double V3 = ((double(*)(double))dlerf)(V2);VMR1
>       (V3);}
> }
> /*    global entry for the function CMP-ANON  */
> 
> static void L1()
> {     register object *base=vs_base;
>       base[0]=make_longfloat(LI1(lf(base[0])));
>       vs_top=(vs_base=base)+1;
> }
> #(#(NIL
>     (%INIT
>      . #((MDL 'erf 'libm 1)
>          (LET ((*DISABLE-RECOMPILE* T))
>            (MF 'CMP-ANON 0)
>            (ADD-HASH 'CMP-ANON '((LONG-FLOAT) LONG-FLOAT)
>                '((erf (FLOAT) T))
> LISPLAMBDA!
!,DECLAR,OPTIMIZ,SAFETY
> libmerf-
>                '/tmp/gazonk_18613_0.lsp))
>          (DO-RECOMPILE)))))
> static void L1();
> static double LI1();
> static void *dlerf;
> #define VMB1
> #define VMS1
> #define VMV1
> #define VMR1(VMT1) return(VMT1);
> #define VM1 0
> static void * VVi[2]={
> #define Cdata VV[1]
> (void *)(L1),
> (void *)(&dlerf)
> };
> #define VV (VVi)
> NIL
> 
> COMPILER>(funcall (compile nil  '(lambda (x) (|libm|:|erf| x))) 1.0)
> 
> ;; Compiling /tmp/gazonk_18613_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> ;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
> (Debug quality ignored)
> ;; Finished compiling /tmp/gazonk_18613_0.o.
> ;; Loading /tmp/gazonk_18613_0.o
>  ;; start address -T 0x9e8b08 ;; Finished loading /tmp/gazonk_18613_0.o
> 0.84270079294971489
> 
> COMPILER>(funcall (compile nil  '(lambda (x) (declare (long-float x)) 
> (|libm|:|erf| x))) 1.0)
> 
> ;; Compiling /tmp/gazonk_18613_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> ;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
> (Debug quality ignored)
> ;; Finished compiling /tmp/gazonk_18613_0.o.
> ;; Loading /tmp/gazonk_18613_0.o
>  ;; start address -T 0xa08948 ;; Finished loading /tmp/gazonk_18613_0.o
> 0.84270079294971489
> 
> COMPILER>(si::save-system "/tmp/h")
> address@hidden:/fix/t1/camm/debian/gcl/tmp/tmp/foo1$ /tmp/h
> GCL (GNU Common Lisp)  2.7.0 CLtL1    Jun 15 2007 19:45:46
> Source License: LGPL(gcl,gmp,pargcl), GPL(unexec,bfd,xgcl)
> Binary License:  GPL due to GPL'ed components: (XGCL READLINE BFD UNEXEC)
> Modifications of this banner must retain notice of a compatible license
> Dedicated to the memory of W. Schelter
> 
> Use (help) to get some basic information on how to use GCL.
> 
> Temporary directory for compiler files set to /tmp/
> 
> >(|libm|:|erf| 1.0)
> 
> 0.84270079294971489
> 
> >
> =============================================================================
> 
> Notes: 
> 
> 0) based on dlopen
> 1) Not yet tested on static linking
> 2) Cannot run such functions interpreted for the moment
> 3) compiling gives both a function with error checking, and an inline
>    providing single instuction access through a C pointer where
>    possible.
> 4) Plan on shipping a little blas, maybe mpi and lapack file to be
>    optionally loaded in the GCL distribution
> 5) package LIB contains libary name symbols bound to the dlopen
>    address of the library
> 6) each library has its own package with symbols bound to the external
>    function address.
> 7) symbols are created and linked on .o load if necessary
> 8) loaded .o code keeps a list of its external pointers in use, which
>    are then reset on image re-execution.
> 
> Enjoy!
> 
-- 
Gernot Hueber <address@hidden>