Re: FFI again

[Stephen J. Turnbull]

Stefan Monnier writes:

 > I personally don't have any experience either way.

The SXEmacs people do, though (using libffi).  Their proof-of-concept
is a simple wrapper for libcurl.  If somebody wants to push forward
with this they should ask.  Most of the other scripting languages
(Python for sure has "ctypes", Ruby has something, maybe Guile does
too) have libffi wrappers.  That experience might be useful too.

Me, I'm just going to use the SXEmacs patch (for XEmacs).  If Steve Y
is willing to sign off, that is probably a good route for Emacs to
start with (the DEFUN macro is different and I don't think ours is
FSF-assigned, but the translation is routine).

 > But I think that a good benchmark for an FFI is to make our libxml
 > and libgnutls code use it.

libxml, maybe.  libgnutls I would continue to provide a dedicated
wrapper for.  It's very tricky to get the Lisp-to-C type matching
robust in an FFI, and if there's a problem in the FFI you probably
break all existing modules by fixing it.  In the meantime, if the FFI
doesn't do *all* the checks just right (and that's very hard to do in
standard C because called libraries typically use platform-dependent
types like "int" rather than "int32", and are often pretty cavalier
about casting internally), you have a crash waiting for some
unsuspecting Lisp code.  Nevertheless, there is a lot of pressure to
avoid changing the FFI.  Better to tweak all that stuff in dedicated
code when you're dealing with security code.

 > > The routines declared in lisp.h do not form stable interface.
 > 
 > Indeed, we'd need to define a slightly more stable API.

This is awfully tricky.  The Python guys spent something like ten
years developing their ABI specification, and had the advantage of a
religious mandate for backward compatibility.  Emacs has always been
wild and woolly with respect to APIs and ABIs since they're all
internal as far as Emacs is concerned.  Even in the published Lisp
Reference backward compatibility with 3rd party programs is frequently
(as measured with decade time units) sacrificed to somebody's sense of
symmetry.  Every release of Emacs has new surprises for XEmacs and
SXEmacs.  (This isn't a complaint, it's a statement of fact.  The
point is that these surprises are much more painful for users at the
level of C code because it's not safe and because it's harder for most
of them to change.)