Re: call/cc and recursive vm invocations

[Ludovic Courtès]

Hi Andy!

Andy Wingo <address@hidden> writes:

> In case you haven't noticed yet, if you get an error at the REPL and ask
> for a backtrace from the debugger, you get not only the frames that
> pertain to your own computation, but also frames from the REPL
> implementation itself. That's not good, and it's a regression.

It’s a regression in ‘make-stack’, right?  Can you remind me what caused
it?

> The solution is to implement start-stack for the VM. Prompts provide an
> ideal set of primitives. So start-stack can be this:
>
>   (define (start-stack tag exp)
>     (let ((prompt-tag (fresh-tag)))
>       (with-fluids ((stacks (acons tag prompt-tag (fluid-ref stacks))))
>         (% prompt-tag
>            exp
>            (lambda (k . args)
>              (apply values k args))))))
>
> This assumes some fluid is around named "stacks". Then we can implement
> make-stack:
>
>   (define (make-stack tag . narrowing)
>     (let ((prompt-tag (assq-ref (fluid-ref stacks) tag)))
>       (unless prompt-tag
>         (error "prompt tag not found quoi"))
>       (narrow-stack
>        (continuation->stack
>         (call/cc
>          (lambda (k) k)
>           prompt-tag))
>        narrowing)))

Looks nice!  :-)

> So, that's the upside. The downside of delimiting "full" continuations
> is that you can't capture the C stack.

[...]

> But of course, to each implementation its own time, and this approach to
> continuations has been nearing its expiration date for years now. Most C
> code can't deal with nonlocal exits, let alone multiple returns. So the
> utility of this construct has been limited.

Indeed.  In libguile, out of 40 ‘scm_dynwind_begin ()’ invocations, only
7 are ‘SCM_F_DYNWIND_REWINDABLE’ (and that doesn’t mean multiple returns
would actually provide a sensible result in those cases.)

[...]

> Anyway, enough about that. Practically speaking, not capturing the C
> stack means that you cannot invoke a continuation that was captured
> outside the current VM invocation.

IIUC, two things could happen (assuming ‘filter’ is a C function):

  1. Failure at ‘call/cc’-time, because the stack contains multiple VM
     invocations intertwined with C function calls.  For example:

       (filter (lambda (x)
                 (call/cc ...))
               lst)

  2. Failure at the time the continuation is invoked, because it’s
     invoked in the context of a different VM invocation than
     ‘call/cc’.  For example:

       (call/cc (lambda (k)
                  (filter (lambda (x)
                            (k ...))
                          lst)))

You were referring to case #2.  Is this correct?

[...]

> The most common causes for recursive incovations for a repl startup
> were, in no order:
>
>   primitive-load
>   load-compiled/vm
>   primitive-load-path
>   call-with-input-string (only once or so)
>   map
>   for-each
>   hash-for-each
>   scm_eval_closure_lookup (?)
>   scm_thunk_p (calls program's meta-thunk to get arity; wasteful GC-wise)
>   scm_resolve_module (calls Scheme resolve-module)
>   filter

That’s for a REPL startup, but we have lots of primitives written in C.
I’d expect a ‘call/cc’ failure to be likely in an arbitrary program.
What do you think?

> Practically speaking... I think I can delimit call/cc with not much work
> (perhaps tomorrow). But it is a visible change (if you're looking), so I
> wanted to put this mail out there to get comments. I had thought this
> was a 2.2 feature, but given the make-stack implications, I'm thinking
> it's a 1.9.9 feature. Reactions?

I’d be rather inclined to wait until 2.2.  While I agree that the
usability of ‘call/cc’ is currently limited for the reasons you gave, I
fear that doing away with the C stack capture may render ‘call/cc’ even
less usable for code that exists, mixes C and Scheme, and has been able
to work around the limitations.

I also think that we should be stabilizing things if we want to release
Real Soon Now, and that 2.2 doesn’t have to wait until 2020 anyway.  :-)

Another question is: can ‘make-stack’ be fixed in the current framework?
This would be less elegant but also less disruptive.

Cheers,
Ludo’.