Re: Grow heap aggressively during music interpretation (issue 561390043 by address@hidden)

[Han-Wen Nienhuys]

On Fri, Feb 7, 2020 at 5:53 PM Han-Wen Nienhuys <address@hidden> wrote:

> Thanks, I ran the carver score successfully now.
>
> I took some pauses between runs so thermal throttling wasn't an issue.
>
> My standard guile2.2 branch (with 40M initial heap), takes about 2m wall
> time, 1m of GC time.
>
> The stats printing shows that we have a very full heap:
>
> In-use heap: 85% (370824 KiB pointers + 60065 KiB other)
> In-use heap: 85% (370725 KiB pointers + 59737 KiB other)
> In-use heap: 85% (370143 KiB pointers + 59687 KiB other)
> ..
>
> so we have to scan a lot of live data to reclaim just 15% of the heap.
>
> Tinkering with INITIAL_HEAP lets us tune things a little bit:
>
>  initial heap 2G: 50s (2.69s GC)
>  initial heap 900M: 0:54 (6s GC)
>  initial heap 500M: 1:09 (33s GC)
>
> The GC timings seem wonky, but since BDW is multithreaded, it's possible
> that the computation times don't completely add up.
>
>

Single threaded, the numbers make more sense:


MAX=INIT=2G
gc time taken: 1.843968805
User time (seconds): 49.36

MAX=INIT=1G
gc time taken: 3.291264925
User time (seconds): 51.74

MAX=INIT=800M
gc time taken: 5.760042906
User time (seconds): 54.62

500M
gc time taken: 17.921457247
User time (seconds): 68.24


It's interesting to note that the multithreaded collector doesn't really
help. With a 500M heap, wall clock is 1:08 for the single threaded case,
and 1:09 for the multithreaded case.

The 800M case seems like a good configuration: it spends about 10% of the
time doing GC.

$ grep In-use log

In-use heap: 0% (0 KiB pointers + 0 KiB other)
In-use heap: 2% (20959 KiB pointers + 2430 KiB other)
In-use heap: 2% (14698 KiB pointers + 2415 KiB other)
In-use heap: 39% (292008 KiB pointers + 29486 KiB other)
In-use heap: 47% (334968 KiB pointers + 53702 KiB other)
In-use heap: 45% (317237 KiB pointers + 53424 KiB other)
In-use heap: 57% (407528 KiB pointers + 67897 KiB other)
In-use heap: 53% (378243 KiB pointers + 60449 KiB other)
In-use heap: 53% (378249 KiB pointers + 60519 KiB other)
In-use heap: 53% (377488 KiB pointers + 60322 KiB other)

so we should scale the heap so that approximately 50% is collected on GC.


-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen