[Chicken-janitors] #1540: Used heap reported by memory-statistics is too big after heap resize

[Chicken Trac]

#1540: Used heap reported by memory-statistics is too big after heap resize
-------------------------------+-------------------------------
 Reporter:  megane             |                 Owner:
     Type:  defect             |                Status:  new
 Priority:  not urgent at all  |             Milestone:  5.1
Component:  core libraries     |               Version:  5.0
 Keywords:  gc heap memory     |  Estimated difficulty:  medium
-------------------------------+-------------------------------
 The reported used heap just after heap resize seems to be `total_allocated
 + last_heap`.

 I would expect the value to be `2 * total_allocated`. In the code below
 this means `(/ total-allocated used)` should be close to `0.5`.

 The actual used memory as repoted by `top` indicate that the problem is
 just with the reported used value; there's no phantom objects on the heap.

 Here's code that I used to test. The report is from C4, but the results
 are similar with C5.
 {{{
 (cond-expand
  (chicken-5 (import (chicken base) (chicken gc) (chicken blob)))
  (else (begin)))

 (define stop-megs 400)
 (define stop-before-resize? #t) ; #f -> stop just after resize

 (define (heap-ref stats) (vector-ref stats 0))
 (define (used-ref stats) (vector-ref stats 1))

 (let ([l '()]
       [total-alloc 0]
       [stats (memory-statistics)])
   (define last-heap (heap-ref stats))

   (let lp ()
     (let* ([cur (memory-statistics)]
            [bytes (max 500
                        (inexact->exact
                         (floor
                          (* .5 (- (heap-ref cur) (used-ref cur))))))]
            [cur-used (used-ref cur)]
            [cur-heap (heap-ref cur)])
       (unless (or (and (< (* stop-megs 1000 1000) cur-used)
                        (and stop-before-resize? (< cur-heap (+ cur-used
 bytes))))
                   (< (* 2 stop-megs 1000 1000) cur-used))

         ;; alloc
         (set! l (cons (make-blob bytes) l))
         (set! total-alloc (+ bytes total-alloc))

         (let* ([stats-after (memory-statistics)])
           (print)
           (print "heap - used = "(- (heap-ref cur) (used-ref cur)))
           (print "ALLOC " bytes " expected used: " (+ (used-ref stats-
 after) bytes)
                  " diff: " (- (used-ref stats-after) (+ (used-ref cur)
 bytes)))
           (print "used / heap: " (used-ref stats-after) " / " (heap-ref
 stats-after)
                  " total-alloc: " total-alloc)
           (print "(/ total-alloc used) = " (* 1.0 (/ total-alloc (used-ref
 stats-after))))
           (unless (= last-heap (heap-ref stats-after))
             (set! last-heap (heap-ref stats-after))
             (print "#################### Heap resized
 ####################"))
           (if (and (not stop-before-resize?)
                    (not (= (heap-ref cur) (heap-ref stats-after)))
                    (< (* stop-megs 1000 1000) cur-used))
               'stop
               (lp))))))

   (print "\nSTOP")
   (let ([cur (memory-statistics)])
     (print "used / heap: " (used-ref cur) " / " (heap-ref cur)
            " total-alloc: " total-alloc)
     (print "(/ total-alloc used) = " (* 1.0 (/ total-alloc (used-ref
 cur))) ))
   (read)
   (print (length l)))


 ;;; (define stop-before-resize? #f):

 ;; heap - used = 696
 ;; ALLOC 500 expected used: 402653572 diff: 84
 ;; used / heap: 402653072 / 402653184 total-alloc: 200922244
 ;; (/ total-alloc used) = 0.498995929677149
 ;;
 ;; heap - used = 160
 ;; ALLOC 500 expected used: 603980700 diff: 201326676
 ;; used / heap: 603980200 / 805306368 total-alloc: 200922744
 ;; (/ total-alloc used) = 0.332664454894382
 ;; #################### Heap resized ####################
 ;;
 ;; STOP
 ;; used / heap: 603980008 / 805306368 total-alloc: 200922744
 ;; (/ total-alloc used) = 0.332664560645524
 ;; 133

 ;; top shows:
 ;; 804724 400360   5716 S   0.0  1.2   0:02.29 gctest


 ;;; (define stop-before-resize? #t):

 ;; ...
 ;; heap - used = 696
 ;; ALLOC 500 expected used: 402653572 diff: 84
 ;; used / heap: 402653072 / 402653184 total-alloc: 200922244
 ;; (/ total-alloc used) = 0.498995929677149
 ;;
 ;; STOP
 ;; used / heap: 402652880 / 402653184 total-alloc: 200922244
 ;; (/ total-alloc used) = 0.49899616761713
 ;; 132

 ;; top shows:
 ;; 411504 400212   5548 S   0.0  1.2   0:02.19 gctest

 }}}

--
Ticket URL: <https://bugs.call-cc.org/ticket/1540>
CHICKEN Scheme <https://www.call-cc.org/>
CHICKEN Scheme is a compiler for the Scheme programming language.