Native Code Again

[Noah Lavine]

Hello all,

Sorry for all of the discussion about native code generation. I have
another idea for how we could implement a good interface to JITed
code. I've actually been thinking it for a while and initially
dismissed it as too crazy, but now I'd like to see what you all think.

The issue this solves that there is really no portable way for C code
to do tail-calls. The ideal method of calling natively-generated code
would be for the VM to tail call the JITed code, which would then tail
call the VM when it was done. The only two ways of doing this are
asking GCC to add a tail call intrinsic (unportable) or building a
library of tail-call ASM for different platforms (more generally
useful, but also basically unportable).

However, there is a third possibility: don't have the VM be C code.
Specifically, you could construct the VM using the JITcode generator,
either on Guile startup or whenever the user decided to enable native
code generation. The benefit of this is that the VM could communicate
with native code using any method our JIT library supports, rather
than any method you can do in portable C. Also, it introduces no new
dependencies than what you need anyway for native code generation,
since you would only need to do this in cases where you would be
generating native code, and in that case you would have to have JIT
capability anyway. The downside is that it's a big new batch of code
to add to Guile.

Just so you know, I have a thought on how to implement this. I posed
here a while back about generating the VM from s-expressions, but I
now have a different idea. I think that Guile should have a C code
parser, if for no other reason than that we could parser C header
declarations and make foreign function interfaces automatically. Given
that, it wouldn't be too much more work to compile C to libjit.
Therefore, I think the path to a native-code VM is to leave the VM as
it is (except maybe reserve an opcode for native-code calls). Then I
write first a C parser for Guile and then a converter program that
would take Guile's current VM and output a JIT VM like I've described.

One downside of this is that libjit probably doesn't have as good
optimization abilities as GCC, so the VM produced might be slower.
Another issue is that this is quite a bit of complexity, but actually
most of the complexity would happen at the time we compile Guile, not
Guile runtime. By the time Guile is running, the only remaining task
should be to run libjit once and go.

So, what do you all think?

Noah