[Chicken-users] A proposal regarding FFI

[Ed Watkeys]

Hi,

I'd like to talk about some of the weaknesses of the current FFI 
interface. Some of the weaknesses may in fact be holes in my knowledge 
of how Chicken's FFI works, and if that's the case, please chime in and 
set me straight.

Gripe number one: the inability to pass to or return from foreign 
functions anything but atoms.

I am disregarding vectors at the moment, since 1) I don't know how to 
get the length of a vector in a foreign function, and 2) they are 
immutable and therefore almost atom-like. Most importantly, I think 
pairs should be passable to foreign functions, and there should be 
C-accessible cons, car, and cdr functions/macros/whatevers.

Gripe number two: the general static-ness of argument passing.

Ideally, there should be some mechanism whereby a C function is passed 
an argument list structure that can then be walked (if positional) or, 
optionally, accessed by key (if keyword-based). There should also be a 
return value structure, passed into a function (by reference) that is 
used to return value(s) along with success/failure/exception 
information.

I see this as an additional method of providing access to foreign 
functions: the existing model works very well for the things it works 
for.

A proposed solution.

I imagine a standard function signature for calls like this. There are 
four FFI-like systems floating around in my head as I write this: 
Chicken's, Gambit-C 4.0's, Objective-C's, and the ANSI C stdard 
facility. Additionally, it seems to me that there's no reason not to 
borrow ideas from models like Java servlets, or the Unix main(argc, 
argv) convention, since what's going on is fundamentally the same in 
all of these cases.

Anyway, here's a first cut of a function implemented with such an 
interface:

// Inspired by THE C PROGRAMMING LANGUAGE, Second Edition,
// Section 7.3. Assumes C99 or C++.

int my_printf(struct scheme_request *req, struct scheme_response *res) {
    scheme_arg_list args;
    scheme_arg *arg;

    scheme_args_start(&args, req, res);

    if(!scheme_next_arg(&arg, UTF8_STRING)) return SCHEME_EX_USAGE;

    // If argument is not a UTF-8 string, Scheme can report
    // the following:
    //   "Function printf expected UTF-8 string
    //    but got <actual-type> for argument 1."
    //
    // If no argument exists, Scheme can report:
    //   "Too few arguments provided to function
    //    printf."
    //
    // Type values can be logical OR'd (or added) together to
    // allow more flexibility. In that case, the actual
    // type of the value can be found in arg->type.
    char *fmt = arg->utf8_string_val;

    for(char *s = fmt; *s; s++) {
        int ival;
        double dval;
        char *sval;
        scheme_pair *pval;

        if(*s != '%')
            putchar(*p);
            continue;
        }

        switch(*++p) {
            case 'd':
                if(!scheme_next_arg(&arg, INTEGER)) return 
SCHEME_EX_USAGE;
                ival = arg->integer_val;
                printf("%d", ival);
                break;
            case 'f':
                if(!scheme_next_arg(&arg, DOUBLE)) return 
SCHEME_EX_USAGE;
                dval = arg->double_val;
                printf("%f", dval);
                break;
            case 's':
                if(!scheme_next_arg(&arg, UTF8_STRING)) return 
SCHEME_EX_USAGE;
                sval = arg->utf8_string_val;
                printf("%s", sval);
                break;
            default:
                putchar(*p);
                break;
        }
    }

    if(!scheme_args_done(&arg)) return SCHEME_EX_USAGE;
    // If there are extra arguments, scheme can report:
    //   "Function printf passed <number> extra
    //    arguments."
    // Or, we could just let it slide and perhaps print
    // a warning to stderr.

    scheme_return_undefined_value(res);
    // If we were returning a value, we'd call one of these, or
    // more than one if you want to return multiple values:
    //
    // scheme_return_integer_value(res, 12);
    // scheme_return_character_value(res, 'e');
    // scheme_return_utf8_value(res, string_val, 
SCHEME_STRING_FINALIZER);
    // scheme_return_utf8_value(res, string_val, my_finalizer);
    // scheme_return_utf8_value(res, string_val, NULL);
    // scheme_return_pair_value(res, pair_val, my_finalizer);
    // scheme_return_opaque_pointer_value(res, opaque_val,
    //                                    type_name, finalizer_value);
    // etc...
    return SCHEME_EX_OK;
}

Discussion.

The above approach mixes the C stdard variable argument approach with 
the Java servlet request/response model with a dash of Unix sysexits.h. 
Here are some advantages:

* It borrows from widely distributed memes.
* The various functions can be implemented either as functions or 
macros.
* It can be used to support Scheme exception throwing by using a result 
of code (e.g. SCHEME_EX_EXCEPTION) and fields in the response object.
* It would allow a reasonably efficient implementation by not 
necessarily requiring dynamic memory allocation for the request and 
response objects.
* It provides a way for foreign functions to tell Scheme about how to 
finalize objects passed to it, and whether Scheme can take 
responsibility for an object.
* It support multiple return values.

Here are some things I haven't addressed:

* Creating and accessing pairs in C land. (Seems trivial.)
* Passing and returning arrays (or hashtables or...) of objects. (Not 
so trivial.)

Yes, I realize these two points are two of the big motivations for 
beginning this exercise, but given what I've written so far, dealing 
with them won't be a big deal.

Suggestions? Opinions?

Ed

--
Watkeys Product Design -- http://watkeys.com/