[Chicken-users] set-finalizer! and cons

[Zbigniew]

I'm in the middle of adding a bunch of stuff to the objc egg, and have
encountered a problem.  For the first time, I happened to write some
code which consed together a few thousand Objective C objects
(#<objc:instance>) into a list.  Suddenly, I took a gigantic speed hit
using cons, as compared to code which simply created, used and
discarded each object.  I tracked this down to set-finalizer!  Here is
a simple loop which creates a finalizer for each list object, then
conses the list objects together.

(define (s n L)
  (if (fx= n 0)
      'done
      (let ((obj (list 1 2 3)))
        (set-finalizer! obj noop)
        (s (fx- n 1)
           (cons obj L)) )))

,t (s 5000 '())
  11.463 seconds elapsed
  11.355 seconds in (major) GC
    2052 mutations
       0 minor GCs
    2951 major GCs

The same loop without the set-finalizer! takes fractions of a second
and 1 major GC.  The finalizers are not actually called yet and don't
count against the total time.

Replacing (cons obj L) with (cons (car obj) L) brings the time back
down to expected.  The same is true if you pass a consing function in,
such as (lambda (x y) (cons (car x) y)), even in the interpreter.  As
demonstrated with my objc egg experience, registering the finalizer
per se does not seem to be the problem.

I mention the consing function because the real code acts very similar
to the example code, but takes kons and knil parameters: it's a
folding function on an NSEnumerator.  In my test, I iterated over an
NSArray containing NSStrings; each NSString will have a finalizer
attached when returned by the bridge.  The performance penalty is
removed if you convert these NSStrings to strings before consing.  But
unexpectedly, the speed up only works for compilation, not from the
interpreter.  And, if your consing function itself incorporates the
string translation---which results in the same ultimate list, a list
of scheme strings---it is just as slow.  For this reason I suspect
holding all these objects in a list is not the actual problem, rather
some threshold is being tripped, and the compiler succeeds at
preventing this only under certain circumstances.

For the curious here is a snippet of the real code; it is probably not
germane to this problem.

;;; ------------- compiled with csc -s -O2

(define (ns:enumerator-fold kons knil e)
  (let loop ()
    (let ((obj (objc:send e next-object)))
      (if (not obj)
          knil
          (kons obj  ;; use (schemify obj) instead of obj to reduce
time to normal
                (loop))))))

(define filename
    (@ @"~/Music/iTunes/iTunes Music Library.xml"
string-by-expanding-tilde-in-path))
(define plist (@ NSDictionary dictionary-with-contents-of-file: filename))
(define tracks (@ plist object-for-key: "Tracks"))
(define e (@ (@ tracks allKeys) object-enumerator))  ;; 5670 entries

;;; ------------- interpreter session

(define L (ns:enumerator-fold cons '() e))     ;; slow unless
(schemify obj) is used inside function
;   17.94 seconds elapsed
;  16.999 seconds in (major) GC
;    3578 major GCs

(define L (ns:enumerator-fold (lambda (x y)
                                (cons (schemify x) y))
                              '() e))                                 
      ;; unexpectedly and equally slow

;; but adding (schemify obj) directly into ns:enumerator-fold, and
compiling, yields
;     0.432 seconds elapsed
; 8.2e-02 seconds in (major) GC
;       17 major GCs