Storing Bytecode Offset

[Zach Shaftel]

Hello all,

Rocky Bernstein and I have been working on the bytecode interpreter
to save the offset as functions are executed. The plan is to then map
the new information to source code locations once that's in place, for
more helpful backtrace information; but first the offset recording needs
to be done right. We've tested[1] a number of different approaches and
are still weighing the pros and cons of them, so we'd like to hear
others' input.

Rocky has pushed the code to Savannah. In the
feature/zach-soc-funcall-from-bytecode branch, I made a crude transplant
of code from Ffuncall and a few others into exec_byte_code, so that
compiled function calls don't need to pass through Ffuncall. In the case
of lexically scoped compiled functions, exec_byte_code is called
directly. The benefit here is that the offset doesn't have to be made
available in the thread struct (it can be passed as an argument to a
record_in_backtrace_with_offset function), and cutting out the Ffuncall
speeds up bytecode execution in general. According to tests, this
version is faster than the existing byte code interpreter, while still
storing the offset for every frame. But the code on that branch is just
a rough proof-of-concept and isn't 100% accurate. Doing it right would
require a lot of refactoring that could ultimately lead to a dead end.

In the feature/zach-soc-bytecode-in-traceback branch, the offset is
stored in the thread_state struct. Prior to the most recent commit, the
offset was then stashed in the backtrace specbinding frame from
record_in_backtrace, so each bytecode call in the backtrace buffer was
annotated with the offset when the error occurred. This is the ultimate
goal, but the current implementation is flawed and a significant source
of slowdown. Even without that, the code is slow, which is why we hope
there are other ideas or ways to improve. We've tried other changes,
like storing the next backtrace frame in the thread_state to eliminate a
loop to find the pertinent frame, but so far we haven't been able to
achieve acceptable performance in this implementation.

-Zach

------------------------------

[1] We've been testing with scripts at
https://gitlab.com/Zach_S/bench-compare.el. We've collected some data in
the `results` and `data` directories. The compare-benches.el script
concurrently runs elisp-benchmarks.el on supplied Emacs executables and
collects all the results into an org-mode file, like the ones in
`results`. If you try it out, let me know what goes wrong, I'm sure
there are bugs.