[PATCH] Add line number tracking to the interpreter

[Peter Bex]

Hi all,

Attached is a set of patches (also to be found in the
"line-numbers-in-csi" in the branch) to add line number tracking to the
interpreter (both for csi and user-invoked (repl) calls).

The first commit simply moves some code around.  Having the line
number database accessors in support.scm makes them only available
to the compiler, while we now want to be able to use them from the
interpreter as well.  So we move it to expand.scm.

The second patch changes the interpreter to track line numbers via
fluid-letting the ##sys#read/source-info-hook, like the compiler does.
In order to store the line number info in the trace buffer, we have
to also extend the trace buffer struct to hold either raw C strings
or Scheme strings for the location.

The third patch changes the alist we store in the line number hash table
to weakly hold onto the form (using the new weak-pairs feature), so
that we can drop unreferenced forms from the database.  I tested this
with a trivial program consisting of two files:

  ;; loader.scm
  (import chicken.syntax)
  (let lp ((i 0))
    #;(when (= 0 (modulo i 999))
      (##sys#display-line-number-database))
    (when (< i 9999999)
      (load "def.scm")
      (lp (add1 i))))

  ;; def.scm
  (define x (lambda (y)
             (+ y 1)))

If you run csi -s loader.scm, this would eat up more and more memory
with only the first two patches, but it stays constant with the third
patch.

The fourth patch improves line number tracking a bit to ensure we
keep line numbers during syntax expansion (like in the compiler).
Without this, you see a lot of un-numbered <syntax> entries in the
csi trace output.

The fifth patch moves the tracking to (repl), so it also works for
user-created REPLs, not just csi.

The sixth patch is a small refactor so we're not using fluid-let to
override the ##sys#default-read-info-hook for (read), but instead
use ##sys#read/source-info directly in all code that will read and then
evaluate code.

I'm only wondering if we need to make the hash table itself weak as
well.  This would require quite a few more changes and I'm not sure of
the benefits.

Cheers,
Peter

0001-Move-line-number-database-accessors-from-support.scm.patch
Description: Text document

0002-Add-initial-support-for-having-line-numbers-availabl.patch
Description: Text document

0003-Convert-line-number-db-hash-table-to-use-weak-alists.patch
Description: Text document

0004-Improve-line-number-tracking-in-interpreter-after-sy.patch
Description: Text document

0005-Move-line-number-tracking-from-csi-to-repl-proper-an.patch
Description: Text document

0006-Don-t-override-sys-default-read-info-hook-to-read-wi.patch
Description: Text document

signature.asc
Description: PGP signature