Re: Future of g-wrap (and guile wrappers in general).

[Rob Browning]

Matthias Koeppe <address@hidden> writes:

> thanks for your extensive comparison between g-wrap and SWIG.  I've
> always wanted to look into g-wrap in detail but never had the time,
> so you've actually saved me a lot of work.

Glad it helps.

> I am going to comment on some of the issues.

Great.

> There is no automatic support for "forced coercions".  If you wanted
> to cast an "int*" to a "double*", you would have to make an
> appropriate identity function and wrap it:

So it sounds like right now there's really no way to explicitly
convert a wrapped pointer of one type to a wrapped pointer of another
arbitrary type without making all types interchangable.  i.e. swig
doesn't have any equivalent to:

  (gw:coerce-wptr foo <gnucash:Bar*>)

If not, then we'd definitely need a workaround or need to stick with
g-wrap for now.

> However, this seemed very inflexible and unnatural to me, so I dropped
> this distinction.  In current SWIG, every C type is handled with
> typemaps, and users are encouraged to use typemaps for their work.
> The typemaps for the "built-in" types are read from the file
> "typemaps.i".  Because typemap definitions tend to be a little
> verbose, I am making use of SWIG's built-in extended C preprocessor
> there.

Way back, my biggest problem with the typemaps was that they weren't
quite flexible enough to do some of the things I wanted.  I suppose
I'll just have to see if all the stuff that's been added is sufficient
if we decide to give swig a go.  I can't recall what the exact issues
were, but I do recall that I needed to insert some code in a
particular place in the generated wrapper, and I couldn't...

> Well, I am using the gh_ functions for converting integer types, and I
> am currently not very careful about the ranges.  This needs to be
> improved.  Since the gh_ API is deprecated, I guess I need to replace
> this by calls to some scm_ functions, doing my own range checking.

Actually, I think there are scm_ range checked conversion functions,
but we need better docs so we can find them easily :>

> The empty list takes this role in SWIG (it's so convenient to type
> "(null? p)" to check for a NULL pointer ;-), but I could easily change
> SWIG to take #f as well.  The behavior can also be changed by the
> user, simply by overriding the typemaps for pointer types. 

As long as there's some way to handle NULL, that's probably
sufficient...

> I would simply make a typedef for those types:
>
>         typedef char *chars_caller_owned;
>         typedef char *chars_callee_owned;  etc
>
> use these type names in the API and give the appropriate typemaps to
> those types.  No problem here.

OK.  That seems functionally equivalent.

> That SWIG uses the "values" of enums and other constants in the
> wrapper, rather than referring to the constants themselves, is a
> quirky feature indeed.  I actually don't know the reason for this.

That's what I was kinda worried about, though I don't know how often
it would actually be a problem.

So how do you access an enum value from the scheme side.  I presume
you don't have to know the relevant integer -- actually I can just go
check the main docs for this unless swig's guile handling varies from
the default...

> I have tried to avoid emitting wrapper code that depends on the Guile
> wrappers, because that would make for a maintenance nightmare.  SWIG's
> output works with Guile 1.3.4 through Guile 1.5.x.  I guess I am
> forced to make an incompatible change when the gh_ interface is no
> longer available, but not earlier.

Hmm, mainly I was just wondering about whether not using guile's
"official" typedef might cause trouble on some machines (i.e. 64-bit,
etc.)

> SWIG has its own (extended) C preprocessor, it is not using the C
> compiler's preprocessing step.  I am using it extensively, and I
> think it is very stable.
>
> I would always use the preprocessor, so that I can use the same
> header file both for the C compiler and SWIG.  I think this is where
> SWIG's power comes from: You just need to maintain one (annotated)
> interface, which defines the API both for C and higher-level
> languages.

But what bothered me was relying on swig's (non-gcc) preprocessor for
swig files, and obviously gcc's when you get around to compiling the C
code.  If they vary *at all* in how they interpret preprocessor
directives, handle math/pragmas/etc.  It seems like you could get some
possibly nasty results.

> However, I have plans to move from this variable-name annotation
> (which is annoying for other reasons as well) to a real type
> annotation.

I would *much* prefer that.

> Some more comments.
>
> 1) I don't like the idea of having a wrapper generator create "low
>    level" procedures and adding an extra level of glue on the Scheme
>    side.  In the project where I use SWIG, I always design a C API
>    that maps to Scheme procedures with a convenient interface; I try
>    to deal with all conversion issues by writing appropriate typemaps.

Right, but in theory g-wrap wanted to see if it could support wrapping
existing api's as well as possible, and in that case, you wouldn't
always have much say over how the API was designed.  So I figured
there would be cases where you might really need some scheme code "on
top".  For example, providing a clean goops interface to some C
library.  g-wrap can't handle that, but it can provide access to the C
lib, and then scheme code can be added to provide a possibly much more
abstract goops interface...

> 4) A foreign function interface like g-wrap that is configurable in
> the native language is a nice thing to have.  However, I think it
> would be most useful if one could import foreign functions
> dynamically in the run time of the Scheme system.  g-wrap, on the
> other hand, creates C wrapper code which need to be compiled and
> linked in.

I've actually toyed with this idea, perhaps even using libffi and
Marius' lightning work to generate the interface wrapper functions "on
the fly" - you just generate them for a type when you need them, and
string them together.  Clever, and possibly for sparsely used API's,
much more resource efficient.

Anyway, it sounds like SWIG now addresses most of my bigger concerns,
but there are some specific things you can do with g-wrap (and we are
doing) that I'll have to see if we can handle (and want to handle) in
an alternate way.

Also, I think this still leaves (as you hint above) as an open
question whether or not there still might be a good argument for a
very tightly tailored, guile-specific API tool.

For example, in we're currently toying with the idea of augmenting
g-wrap so that it can generate xml-rpc stubs.  It's unclear how hard
that might be, but it's interesting...

-- 
Rob Browning
rlb @defaultvalue.org, @linuxdevel.com, and @debian.org
Previously @cs.utexas.edu
GPG=1C58 8B2C FB5E 3F64 EA5C  64AE 78FE E5FE F0CB A0AD