[Gcl-devel] Re: proclaim problem simplified

[Camm Maguire]

Greetings, and thanks!

Here is what I have locally, almost ready to commit:


>(defun bar () (zero))

BAR

>(compile 'bar)

;; Compiling /tmp/gazonk_24037_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_24037_0.o.
Loading /tmp/gazonk_24037_0.o
start address -T 0x8731ab8 Finished loading /tmp/gazonk_24037_0.o
#<compiled-function BAR>
NIL
NIL

>(load "/tmp/foo")

Loading /tmp/foo.o
add-recompile BAR ZERO NIL (NIL (INTEGER 0 0))
recompiling user::bar because callee user::zero changed sig from lisp::nil to 
(lisp::nil (lisp::integer 0 0))
;; Compiling /tmp/recompile.lsp.
; (DEFUN BAR ...) is being compiled.
;; Warning: ret type mismatch in auto-proclamation (*) -> (INTEGER 0 0)

removing recompile of BAR
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/recompile.o.
Loading /tmp/recompile.o
start address -T 0x87319f0 Finished loading /tmp/recompile.o
start address -T 0x872e2c8 Finished loading /tmp/foo.o
44

>(bar)

0

>
%
>(load "/tmp/foo")

Loading /tmp/foo.o
start address -T 0x877b940 Finished loading /tmp/foo.o
44

>(defun bar () (zero))

BAR

>(compile 'bar)

;; Compiling /tmp/gazonk_24083_0.lsp.
;; End of Pass 1.  
;; End of Pass 2.  
;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
(Debug quality ignored)
;; Finished compiling /tmp/gazonk_24083_0.o.
Loading /tmp/gazonk_24083_0.o
start address -T 0x872e1a8 Finished loading /tmp/gazonk_24083_0.o
#<compiled-function BAR>
NIL
NIL

>(bar)

0

>


A few items, among likely many more :-)

1) user interface needs some work -- what diagnostics should be
   output, how much control is desired, etc.  

2) The trickiest part is to save the compilation environment in a
   truly portable function source saved on the plist for compilation
   at a later time.  For example, eval-when compile macros need to be
   expanded in the saved source, etc.  I'm still chasing down all of
   this, but currently the eval-when compile macros and the ambient
   safety setting are supported, the latter via an addition into the
   source as an explicit (declare (optimize (safety ...))).
   gcl_cmpwt.lsp actually loaded gcl_fasdmacros.lsp in eval-when
   compile, leaving the saved functions without the fasd defstruct
   present to run let alone compile!  I've tried to leave the source
   as unmodified as possible to check the robustness of the new
   system, but it appears there might always be some possible
   compile-time/load-time incompatabilities left no matter what we do.
   Thoughts on this most appreciated.

3) Right now we synchronize a si::*call-hash-table* lookup of the
   function signature with the older proclaim ftype method which
   stores 3 elements on the symbol plist: proclaimed-function
   proclaimed-arg-types and proclaimed-return-type.  I suggest we lose
   the latter and rather use the former to lookup the signature in the
   compiler, but this also needs discussion.  Having two methods
   invites synchonization problems.

4) Recursive functions now undergo a loop over the auto-proclamation
   step until the signature is constant. (starting with a nil return
   type).  Eventually if we ever reverse infer the argument types,
   we'll have to start these at nil too.

5) For compatability with compiler::link, si::*disable-recompile* can
   delay the recompilation step until the base system modules have
   been initialized in the 'raw' image.  When reenabled,
   si::do-recompile sorts the functions needing recompilation
   callee-first, saves the source to ../unixport/gcl_recompile.lsp,
   executes (load (compile-file ...)), and iterates (appending to
   gcl_recompile.lsp) until the *needs-recompile* array has length 0.
   Then a second pass to the system linker ld can link in the new
   gcl_recompile.o and initialize it as usual.  The system comes up
   cleanly on systems where native object relocation is absent.  This
   having been said, getting all these dependencies right will likely
   take a little while, and really could use an insightful suggestion
   as to the proper design/architecture.

...
add-recompile |(PCL::FAST-METHOD PRINT-OBJECT (INTERNAL-ERROR T))| ERROR NIL ((T
add-recompile |(PCL::FAST-METHOD PRINT-OBJECT (INTERNAL-SIMPLE-ERROR T))| ERROR 
NIL ((T
add-recompile |(PCL::FAST-METHOD PRINT-OBJECT (INTERNAL-SIMPLE-WARNING T))| 
ERROR NIL ((T
Initializing gcl_clcs_kcl_cond.o
add-recompile CLCS-UNIVERSAL-ERROR-HANDLER ERROR NIL ((T *) *)
Warning: LOAD is being redefined.
Warning: OPEN is being redefined.
Initializing gcl_clcs_top_patches.o
Initializing gcl_clcs_install.o
add-recompile CLCS-COMPILE-FILE ERROR NIL ((T
add-recompile CLCS-COMPILE ERROR NIL ((T *) *)
...
Initializing gcl_recompile.o
removing recompile of COMPILER::BINDING-DECLS-NEW1
removing recompile of SYSTEM:DEFMACRO*
removing recompile of COMPILER::C1DECL-BODY


6) size -- we have some tools to compress the source into fasd form,
   and to eq hash the signatures, but for a while this is likely to be
   a bit large.  A *complete* flush should be possible whenever the
   system is in a consistent state, (which is intended to be after
   each load).

7) compilation speed -- not too bad, but you can expect ....   We can
   store the hash table when we ship the final source if desired.

8) Having a comprehensive hash table of callers and callees is
   convenient in a number of ways, e.g. finding abandoned/obsolete
   functions, etc.  I'm amazed at how quick these operations can be.
   For example, sorting needs to build a recursivelly-complete list of
   callees, which is exponential -- memoization to the rescue:

(defun all-callees (x y)
  (or
   (union (gethash x *ach*) y)
   (setf (gethash x *ach*)
         (let* ((z (call-callees (gethash x *call-hash-table*)))
                (l (set-difference z y))
                (y (union y z)))
           (dolist (q l y);FIXME
             (setq y (the list (all-callees q y))))))))

...
      (clrhash *ach*)
      (sort *needs-recompile* 
            (lambda (x y) 
              (let* ((x (car x))
                     (y (car y)))
                (member x (all-callees y nil)))))


9) A more compact selective/tunable recursive macroexpander has come
   out of this too:

(defconstant +foo-list+ '(no-atom let let* lambda the flet labels macrolet 
declare quote function))

(defun eewcom (form no &aux fd)
  (cond ((atom form) form)
        ((unless no 
           (when (setq fd (cadr (assoc (car form)  
*eval-when-compile-only-macros*)))
             ;or other macros here
             (format t "expanding ~s~%" form)
             (setq form (funcall fd form nil))
             nil)))
        ((cons (eewcom (car form) (unless (eq (car no) 'no-atom) no))
               (eewcom (cdr form) (or (member (car no) '(declare quote))
                                      (member (car form) +foo-list+)
                                      (when (atom (car form)) +foo-list+)))))))


Just passed the ansi-test suite with 888 failures, about as usual,
with the new system.  Will ensure maxima works before committing.
Will be out of town for the next three days -- may have to wait until
I return.

Take care,

Robert Boyer <address@hidden> writes:

> For the little it's worth, here is a simplified script for
> hitting that auto-proclaim bug.
> 
> Bob
> 
> -------------------------------------------------------------------------------
> 
> % cat foo.lisp
> (proclaim '(ftype (function nil t) zero))
> (DEFUN ZERO NIL 0)
> 
> % xg
> GCL (GNU Common Lisp)  2.7.0 ANSI    May 21 2006 18:10:06
> ...
> >(compile-file "foo.lisp")
> 
> % xg
> GCL (GNU Common Lisp)  2.7.0 ANSI    May 21 2006 18:10:06
> ...
> >(load "foo")
> 
> Loading foo.o
> start address -T 0xacef850 Finished loading foo.o
> 44
> 
> >(defun bar () (zero))
> 
> BAR
> 
> >(compile 'bar)
> 
> ;; Compiling /tmp/gazonk_14062_0.lsp.
> ;; End of Pass 1.  
> ;; End of Pass 2.  
> ;; OPTIMIZE levels: Safety=0 (No runtime error checking), Space=0, Speed=3, 
> (Debug quality ignored)
> ;; Finished compiling /tmp/gazonk_14062_0.o.
> Loading /tmp/gazonk_14062_0.o
> start address -T 0xace9b40 Finished loading /tmp/gazonk_14062_0.o
> #<compiled-function BAR>
> NIL
> NIL
> 
> >(bar)
> 
> Error in EVAL [or a callee]: Arg or result mismatch in call to  ZERO
> 
> 
> 

-- 
Camm Maguire                                            address@hidden
==========================================================================
"The earth is but one country, and mankind its citizens."  --  Baha'u'llah