bug#46900: 28.0.50; GC uses excessive amounts of stack

[Pip Cet]

This is a long-standing issue.

Currently, GC is written to recurse when marking objects. That means
for every level of recursion that we see, a new function stack frame
is being created on the C stack; on this x86_64-pc-linux-gnu system
running Debian GNU/Linux, in an optimized build, that means 80 bytes
of stack space are reserved for each level of recursion.

That number could be much closer to 8, which is the number of bytes of
relevant data in those 80 bytes of stack space.

My proposal is to continue allocating the mark queue on the stack (not
the heap), but to do so in larger chunks, allocating enough space for
N objects and recursing, in the C sense, only when all N object slots
have been used.

In addition to saving stack space, this should also help, at least,
older systems with limited return address prediction stacks.

IMHO, it would also help, not hinder, debugging of GC internals, as
more context would be available in the function frame. Indeed, the
patch I will send once this has a bug number keeps all of the large
"soft" stack frames in a linked list so we can easily walk them.

In further patches, I'd like to change things so we no longer recurse
through symbols so enthusiastically, building up tens of thousands of
objects on the mark queue.

In the interest of full disclosure, I have several ulterior motives
for this: the WebAssembly implementation I'm using doesn't allow deep
nesting of calls; I'd like to look into parallelizing GC for which
this might be useful; and I suspect native-comp will benefit more from
further work in this area, too.