Re: [Chicken-users] #<invalid forwarded object> using FFI callback

[.alyn.post.]

Good day pulijzer.

I have not.  My best guess currently is that my call to make-string
returns a scheme object back to C.  I then make another call to
make-string, and the GC doesn't (so far as I understand it) have a
reference to that first make-string.  If any of these intermediate
make-string calls triggers a GC, I'll be left with a reference that
was garbage collected.  (I'm further given to understand I get a
categorically distinct GC (a minor GC?) that doesn't longjmp,
because I'm calling in to Scheme from non-reentrant C code.  Which
may or may not matter...)

If the GC takes care to mark every object passed to C as a root
pointer this wouldn't happen.  I'm neither sure this is what is
causing my invalid forwarded object problem nor whether or not
Chicken does object marking around the C FFI boundary.

on IRC I was advised to break up my call to C in to smaller pieces
so I wasn't holding any/as many scheme objects.  I haven't revisited
this code since receiving that advice as I find it vaguely
unsuitable and plan to review the relevant parts of the FFI
interface for a more informed fix--as this becomes a crisis rather
than an intermittent problem.

I'm embarrassed to report this code fires during a consistent point
during initialization of my code, and I set a high-enough initial heap
to not trigger a GC.  That initial heap size is there for other
reasons--but it helped make this problem go away for me too.

-a

On Thu, Dec 19, 2013 at 09:15:21PM +0100, pluijzer . wrote:
>    Hello Alyn,
> 
>    Did you find a solution to this problem? I seem to suffer a similar
>    problem.
> 
>    thanks,
>    Pluijzer
> 
>    2013/12/8 .alyn.post. <address@hidden>
> 
>      Greetings,
> 
>      I'm encountering the following error message inside a callback using
>      Chicken's FFI interface:
> 
>      Error: (cdr) bad argument type: #<invalid forwarded object>
> 
>      * * * * Call history:
> 
>      * * * * pwent.scm:100: loop
>      * * * * pwent.scm:56: ##sys#gc
>      * * * * pwent.scm:56: g42
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:50: clist-append!
>      * * * * pwent.scm:100: loop
>      * * * * pwent.scm:56: ##sys#gc
>      * * * * pwent.scm:56: g42
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:24: make-string
>      * * * * pwent.scm:50: clist-append! * * * * * * <--
> 
>      What seems to be happening is that I'm running out of (stack?) memory
>      while inside/around a C thunk. *I cannot seem to avoid this message by
>      modifying the nursery, either from csc -nursery or from -:s.
> 
>      I have only a topical grasp of Chicken's FFI, so I suspect I'm simply
>      doing something ill-advised, though I would appreciate advice on how
>      to approach my problem:
> 
>      I'm trying to read /etc/passwd using getpwent(3) and store each
>      record returned in a scheme list. *(My particular /etc/passwd file
>      contains <100 elements, the above error happens near the end of the
>      call.) *I'm marshalling records using two callbacks: one to create the
>      memory for string data, and one to append each record to the list.
> 
>      May I have feedback on the pattern I'm using here to marshall data?
>      And a suggestion on how to avoid invalid forward objects while
>      reading large but disjoint data from C? *I'm at a loss as to why
>      exactly I'm getting the above message, and it well could be incorrect
>      code on my part. *Do my multiple calls to _make_string from C,
>      below, cause the GC to loose track of my string pointers? *Something
>      else?
> 
>      <++> csc -o pwent pwent.scm && ./pwent
>      (use extras)
> 
>      ; a circular list (where we track the
>      ; head and tail) with a dummy head.
>      ;
>      (define (make-clist)
>      * (let ((head (list #f)))
>      * * (cons head head)))
> 
>      ; O(1) list append.
>      ;
>      (define (clist-append! d v)
>      * (let ((l (list v)))
>      * * (set-cdr! (cdr d) l)
>      * * (set-cdr! d l)))
> 
>      ; return proper list
>      ;
>      (define (clist->list d)
>      * (cdr (car d)))
> 
>      ; allocate a scheme string available in C.
>      ;
>      (define-external
>      * (_make_string (size_t n)) scheme-object
>      * * (make-string n))
> 
>      (declare (foreign-declare #<<EOF
>      #include <sys/types.h>
>      #include <string.h>
>      #include <pwd.h>
>      EOF
>      ))
> 
>      (define (getpwent)
>      * ; append! each pwent record to our record list.
>      * ;
>      * (define-external
>      * * (_getpwent_cb (scheme-object clist)
>      * * * * * * * * * (scheme-object user)
>      * * * * * * * * * (scheme-object passwd)
>      * * * * * * * * * (scheme-object uid)
>      * * * * * * * * * (scheme-object gid)
>      * * * * * * * * * (scheme-object home)
>      * * * * * * * * * (scheme-object shell)) void
>      * * * (let ((pwent `((user * . ,user)
>      * * * * * * * * * * *(passwd . ,passwd)
>      * * * * * * * * * * *(uid * *. ,uid)
>      * * * * * * * * * * *(gid * *. ,gid)
>      * * * * * * * * * * *(home * . ,home)
>      * * * * * * * * * * *(shell *. ,shell))))
>      * * * * (clist-append! clist pwent)))
> 
>      * ; retrieve the next pwent record and marshall it
>      * ; in to scheme.
>      * ;
>      * (define _getpwent
>      * * (foreign-safe-lambda* bool ((scheme-object clist)) #<<EOF
>      * * struct passwd *pw;
>      * * C_word shell, dir, passwd, name;
>      * * size_t n;
> 
>      * * pw = getpwent();
> 
>      * * if(!pw) {
>      * * * endpwent();
>      * * * C_return(0);
>      * * }
> 
>      * * n = strlen(pw->pw_name);
>      * * name = _make_string(n);
>      * * C_memcpy(C_c_string(name), pw->pw_name, n);
> 
>      * * n = strlen(pw->pw_passwd);
>      * * passwd = _make_string(n);
>      * * C_memcpy(C_c_string(passwd), pw->pw_passwd, n);
> 
>      * * n = strlen(pw->pw_dir);
>      * * dir = _make_string(n);
>      * * C_memcpy(C_c_string(dir), pw->pw_dir, n);
> 
>      * * n = strlen(pw->pw_shell);
>      * * shell = _make_string(n);
>      * * C_memcpy(C_c_string(shell), pw->pw_shell, n);
> 
>      * * _getpwent_cb(clist,
>      * * * * * * * * *name,
>      * * * * * * * * *passwd,
>      * * * * * * * * *C_fix(pw->pw_uid),
>      * * * * * * * * *C_fix(pw->pw_gid),
>      * * * * * * * * *dir,
>      * * * * * * * * *shell);
>      * * C_return(1);
>      EOF
>      ))
> 
>      * ; loop ever every entry in pwent and append
>      * ; it to our list.
>      * ;
>      * (let loop ((clist (make-clist)))
>      * * (if (_getpwent clist)
>      * * * * (loop clist)
>      * * * * (clist->list clist))))
> 
>      (pretty-print (getpwent))
>      <-->
> 
>      Thank you,
> 
>      -a
> 
>      $ csc -version
>      (c) 2008-2013, The Chicken Team
>      (c) 2000-2007, Felix L. Winkelmann
>      Version 4.8.0.5 (stability/4.8.0) (rev 5bd53ac)
>      openbsd-unix-gnu-x86 [ manyargs dload ptables ]
>      compiled 2013-10-03 on aeryn.xorinia.dim (Darwin)
>      --
>      my personal website: [2]http://c0redump.org/
> 
>      _______________________________________________
>      Chicken-users mailing list
>      address@hidden
>      [4]https://lists.nongnu.org/mailman/listinfo/chicken-users
> 
> References
> 
>    Visible links
>    1. mailto:address@hidden
>    2. http://c0redump.org/
>    3. mailto:address@hidden
>    4. https://lists.nongnu.org/mailman/listinfo/chicken-users

> _______________________________________________
> Chicken-users mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/chicken-users


-- 
my personal website: http://c0redump.org/