Re: Thoughts on getting correct line numbers in the byte compiler's warning messages

[Alan Mackenzie]

Hello again, Stefan.

On Wed, Nov 07, 2018 at 12:25:15 -0500, Stefan Monnier wrote:

> >> That accumulates for every data type, and it increases code size,
> >> reduces cache hit rate...

> > No, it applies mainly to FIXNUM, because XFIXNUM doesn't already check
> > the Lisp_Type.  Other object types already perform this check, so while

> I'm not sure why you say that.  XCONS/XSYMBOL don't perform the check
> either (unless you compile with debug-checks, of course, but that's not
> the important case).

Ah, really?  OK, I'd need to repeat the exercise with the checks in
XCONS and XSYMBOL, too.  I suspect the slowdown would be significant,
though perhaps not critical (say, around 5%).  For these #defines, there
must be a check on Lisp_Type somewhere, so we should be able to
incorporate that "somewhere" into the check for Lisp_Type 1.  Maybe.

[ .... ]

> There's indeed a pretty good set of bad options at hand.  Not sure which
> one will suck less.

Yes.  Things aren't looking good.

[ .... ]

> It's "only" the cconv-convert part of cconv.el that will need changes,
> but yes, one way or another it will need to be changed to preserve the
> location info.

OK.  But it's still a challenging job.

> > Maybe it would be possible to defer cconv.el processing till after macro
> > expansion and byte-opt.el stuff.  Would this do any good?

> It's already done after macro expansion (but before byte-opt).
> I don't think it moving it would help.

Maybe not.  I was thinking that if it was deferred until after byte-opt,
"all" the warning messages would have the right position info.  But
cconv.el calls byte-compile-warn, too.

> > The only vague idea I have for saving this, and I don't like it one bit,
> > is somehow to redefine \` (and possibly \,) in such a way that it would
> > somehow copy the source position from the original list to the result.

> Define "original list" ;-)

The one that has been transformed into the result.  For example, in this
fragment from the end of cconv-convert:

    (`(,func . ,forms)
     ;; First element is function or whatever function-like forms are: or, and,
     ;; if, catch, progn, prog1, prog2, while, until
     `(,func . ,(mapcar (lambda (form)
                          (cconv-convert form env extend))
                        forms)))

, the original list would be the whole FORM.  My idea would be to
rewrite the resulting form as something like:

    `(form ,func . ,(bc-mapcar (lambda (form)
                                 (cconv-convert form env extend))
                               forms))

, where the first argument in the modified \` supplies the position
information for the result list, but isn't included in the list itself.
bc-mapcar would be a version of mapcar which preserves the internal
position info in the resulting form, copying it from the original list
parameter.

As I say, I don't like the idea, but it might be the best we can come up
with, and still have a readable and maintainable cconv.el.

[ .... ]

> > I've been through these sort of thoughts.  That idea would be less
> > effective than the "extended object", since it would only work with
> > conses, but might be less disruptive.  But why should it only work
> > with conses?

> No particular reason at first.

> > Why not with symbols, too?

> Reproducing this idea for other types is not always that easy or useful:
> - for pseudo-vectors the variable size aspect makes it harder to handle
>   (tho not impossible).  OTOH we could probably use a bit in the header
>   and thus avoid the need to place those extended objects in their
>   own blocks.

Yes.

> - for symbols the extra info is "per symbol occurrence" rather than "per
>   symbol", so we can't add this info directly to the symbol (i.e. the
>   same reason the hash-table approach doesn't work for symbols).

D'oh!  Of course!

>   So we'd really want a completely separate object which then points to
>   the underlying symbol object.  But yes, we could introduce a new
>   symbol-occurrence object, along the lines you originally suggested but
>   only for symbols (thus reducing the performance cost).

:-)  This could be a pseudovector, leaving Lisp_Type 1 free for more
worthy uses.  You're suggesting a mix of approaches.  This might be more
complicated, but possibly the least pessimal.

> -- Stefan

-- 
Alan Mackenzie (Nuremberg, Germany).