Re: Thoughts on getting correct line numbers in the byte compiler's warning messages

[Alan Mackenzie]

Hello, Stefan.

On Tue, Nov 06, 2018 at 15:04:51 -0500, Stefan Monnier wrote:
> >> > Many of the original forms produced by the reader survive these
> >> > transformations.
> >> Yeah, that's why I thought of using a hash-table.
> > What I tried before (about two years ago) was having each
> > reader-produced form as a key, and the source position as a value.  Each
> > time the source was transformed, the new form became a new key, and the
> > value stayed the same.
> >
> > I vaguely remember this being slow.

> Which part do you remember being slow (e.g. just performing a `read`
> that returns a sexp and fills that table along the way)?

Looking at notes I made at the time, I amended a small portion of e.g.
byte-optimize-body to make a new hash entry with the same value when a
form was transformed.  The slowdown on just the byte optimiser was
around a factor of three.  I think the comparison was with the
byte-optimiser in the released version (without any hash tables).

> > Maybe it would be better the other way around.  The source position
> > would be the key, and the value would be a list of (equivalent) forms.
> > Building this table would be faster.

> I don't follow you: why would this be faster?

I don't think I follow myself here.  I was thinking that accessing a
hash table element was slow, therefore keeping a table value current and
pushing transformed forms onto it would be faster than creating a new
hash table entry for these new forms.  Looking at the code for hash
tables, the access time can not be all that long.

> > Finding a form in that table for a warning message would be much
> > slower, but that shouldn't matter.

> It could matter, but yeah, let's not worry about that for now.

> > In byte-compile-warn, if we can't find the current form in the above
> > table, we search for the containing form, get its source offset, put
> > point there and read the next N forms, moving forward in the source text
> > to the position we need.  That this might be slow (I don't really think
> > it would be) is again unimportant.

> I lost you here as well: how is the location data propagated from the
> reader to the byte-compiler's phase that ends up running
> byte-compile-warn?

For objects created by the reader, they can be looked up in the hash
table.  But your real question ....

> I mean, how is the location info preserved while going through
> macro-expansion, closure-conversion, and byte-optimize-form?  Or are
> most objects left untouched in practice?

Either by making new entries in the table for transformed forms, or by
noting byte-compile-containing-form and "sub-form number 2" and using
read (or forward-sexp, even) on the source text to move forward to
sub-form 2.

> I guess we could limit the info (e.g. stored in a hash-table) to map
> "first cons-cell in a list" to its location info, and then change
> macroexp.el, cconv.el, and friends to preserve this info as much as
> possible (we may even come up with a `with-location-data` macro that
> encapsulates most of the work so the changes are easy to apply).

> Is that what you're thinking of?

That's the sort of thing, yes.
>         Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).