[Gcl-devel] auto-recompilation

[Camm Maguire]

Greetings!

OK, my stuff is now in.  Comments most welcome.  Please ignore gcl
build time issues at the moment.

Limitations:

        1) no compiler::link support yet
        2) no adequate tempfile support yet -- working on a solution
        akin to Bob's proposal.  This means that two gcl processes on
        the same box can collide on the recompilation temporary source
        file at the moment.
        3) recursive function type inferencing needs some work
        4) call trees pertain to compiled functions only at present --
        no interpreted function integration/support.
        5) possible proclaim ftype/new call signature synchronization
        problems (none found thus far.)  Eventually will likely remove
        proclaimed-??? symbol-properties if this works out.
        6) (setf function) names need a little work.
        7) supported type granularity could be improved.  Right now we
        bump the types to certain useful and common super types.

How to turn it off:

(setq si::*disable-recompile* t)

How it works:

The guts are in the new file lsp/gcl_callhash.lsp.  Comments most
welcome.


si::*call-hash-table* maps function symbols to a call structure:

             (defstruct (call (:copier copy-call)
                              (:predicate call-p)
                              (:constructor make-call))
                              sig callees callers src)
        
sig is the conventional signature '((args) ret).  callees list the
direct callees, likewise the callers.  src contains portabilized
(recompilable) source in fasd string format, which has been expanded
to support string input and output streams.  (si::function-src 'name)
does what you think it should :-), except that it needs some work in
function-lambda-expression portabilizing interpreted functions of the
4 different types, lambda, lambda-block, lambda-closure, and
lambda-block-closure.  (si::all-callees 'name nil) does what you think
it should too. :-).  

The compiler adds an entry for each callee relationship in
c1symbol-fun, then adds the signature in t1defun when
auto-proclaiming.  The whole call entry is added at load time via the
.data file, as written by t2defun.  clear-compiler-properties has been
rewritten to modify the entries when one does for example (setf
(symbol-function 'g) (symbol-function 'h)).  

Whenever a signature mismatch is detected during these operations, a
(list name callee-causing-conflict assumed-sig actual-sig) is added to
si::*needs-recompile*.  (si::do-recompile &optional (pn
"/tmp/recompile.lsp" pnp)) sorts this array callee first, writes the
source to the specified filename, compiles and loads.  The load will
remove the entry in *needs-recompile* unless the signature still
mismatches.  do-recompile then recurses until *needs-recompile* is
empty. (si::do-recompile) is written at the end of each .data file.

unixport/makefile makes each image twice.  si::*disable-recompile*
starts out at t, and is set to nil in the init-file right after
user-init, causing the recompile into gcl_recompile.lsp.  The image is
then relinked with the newly recompiled source and initialized again,
which should see no recompilation.

There are some boring bootstrapping details if anyone is interested.

There is a small ansi-test issue around read-sequence not yet tracked
down.  Structure accessors and predicates carry a signature but no
source as they are compiled closures -- hence they cannot be
recompiled.  Rudimentary support exists for looking up a function name
from the function (si::function-name).  This is used in an enhanced
documentation function, but not yet in clear-compiler-properties,
where it should replace the simple traversal of the hash-table looking
for a match.  The whole scenario is geared toward function calls by
name, as this is the source of the signature problem to begin with --
anonymous functions cannot be redefined out from under the compiled
caller as far as I can see.

maxima, nqthm, and most ansi-tests together with self-build seems to
be working, at least with --enable-debug which is what I'm using now.

Comments/suggestions always most welcome.  Especially if any of you
feel this is going in the wrong direction.  Continuing on this path
will eventually eliminate (proclaim (ftype)) and sys-proclaims.lisp,
but we can still backtrack if desired.

Take care,
-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah