Re: byte-compile making erroneous *Compile Log*

[David Vanderschel]

David, regarding edebug-eval-defun on fill-region from filladapt.el:
>>   It fails.  So does eval-defun.  The problem is
>>   that, in the documentation string for the
>>   function, there is a `(' in the first column.
>>   I've run into this before.  The function will
>>   compile properly, but evaling the defun
>>   interactively is problematic and the symptoms can
>>   get very weird.  Because of this, I long ago took
>>   to formatting documentation strings with a space
>>   in column 1, for otherwise filling could
>>   inadvertently result in a `(' in col. 0.

Martin:
> I usually put a backslash in front of any parentheses within
> doc-strings so they won't interfere with filling.

I want them to be subject to fill.  I just don't want
them winding up in col. 0, so I have a 1-blank
fill-prefix throughout all my doc strings that might
contain parens.  The symptoms resulting from a `(' in
col. 0 are serious and bizarre.  The reported error
depends on where in the defun point lies when you
request the eval.  Strangely, if you go to the end of
the defun and do eval-last-sexp, it works.
This problem has existed for years.

>>   I discovered that the instance of fill-region that
>>   find-function found is actually an old one in
>>   filladapt.el in my private lisp directory.  (Goes
>>   back to 2002.  Not sure why I had my own copy.)
>>   So I figured that was the problem.  After I fixed
>>   it so that find-function was finding the right one
>>   in the emacs lisp directory, I was dismayed to
>>   discover that it _still_ does not work.  Indeed the
>>   `(' in col. 0 still exists and still causes
>>   problems.  But the wrong behaviour in
>>   *Compile Log* continues; so it was not the fault
>>   of my old private copy.  (Yes, I also hid the .elc.)

> But you probably didn't evaluate the version in the lisp directory so
> you were still using the one with the bad doc-string.  

No.  I reloaded Emacs after I removed filladapt from
my local directory.  The _latest_ version (i.e.,
distributed with v23 emacs-snapshot) still has a `('
in col. 0.  As I said, it will compile OK that way.
When I did find-function, I could see in which
directory lay the file:

filladapt.el -> /usr/share/emacs/site-lisp/emacs-goodies-el/filladapt.el

This is from emacs-snapshot - v23.0.91.1 - straight
from Debian.  The following are the first few lines of
the defun:

______________________________________________________________________
 
(defun fill-region (beg end &optional justify nosqueeze to-eop)
   "Fill each of the paragraphs in the region.

(This function has been overloaded with the `filladapt' version.)

Prefix arg (non-nil third arg, if called from program) means justify as well.

______________________________________________________________________

You can see the troublesome open paren in there.

>Don't care about byte-compiling when doing these
>things.  Use `eval-buffer' instead, it will also give
>you less headaches while debugging.

Indeed.  I also use eval-defun and eval-region
depending on what I have fooled with.

>>   (If you ask to what key is bound eval-defun, it
>>   says \C-\M-x and \M-X, but those are really
>>   bindings for edebug-eval-defun.)

> C-M-x is aliased to `edebug-eval-defun' which calls `eval-defun' when
> there's no prefix argument.  What is "\M-X"?

shift meta x.  I see now that it is a private binding
I made because \C-\M-x conflicted with a global
keybinding I had done in Windows (and eventually
abandoned in favor of global bindings based on the
Windows key, since I steered clear of the Super-
modifier in Emacs).  I have now adapted my
initialization for Ubuntu; and that key binding is not
harmful.

I was confused because in Emacs v20, there really was
a separate function edebug-eval-defun provided by
edebug itself.  (However, it also overrode eval-defun
so that the prefix trick did work there also.)

>> The reason that some "In <defun-name>:  "
>> strings do not get merged with the following line is
>> that fill-region is _not_ called in those cases.
>> Looking at the code for display-warning, it appears
>> that fill-region is called only when
>> warning-fill-prefix is not nil (its default).  But I
>> did not see any code that could change it, so I don't
>> know what causes that - especially dynamically during
>> the compilation.

> Mabye there is a "\n" in MESSAGE already and the
> code is skipped?

Now I have run the debugger in display-warning.  And a
\n is the reason for skipping those that are skipped
and which come out OK.  Some of the warnings do have
\n's in them.  (I guess display-warning figures that,
if the message already contains \n's, then it must have
already been formatted with correct filling and that
it needs no filling.)

The most common sort of warning that is causing the
symptom is assignment to a free variable.  (Some
of those are unavoidable as far as I can tell.)

It strikes me that it is basically wrong for
display-warning to include the "In <defun>: " line in
the region it passes to fill-region.  Furthermore, I
don't see how it is happening because display-warning
sets its start variable (for the beginning of the
region to fill) to (point) before inserting the
message.

>> ... It turns out that
>> paragraph-ignore-fill-prefix was the cause after all.
>> I don't know what went wrong with my test when I
>> thought I had changed it to nil.  But when it is t,
>> the compile log gets fouled up.  I can live with nil.
>>
>> Thanks, Martin, for the pointers.  Do I need to report
>> the bug, or are you pursuing it?

> I don't know because I can't reproduce it yet.  Can
> you reproduce it with emacs -Q and
> `paragraph-ignore-fill-prefix' set to t?

Yes.  But there is something strange here.  If I do
emacs -Q and (setq paragraph-ignore-fill-prefix t), it
still works correctly.  But if I do 
(custom-set-variables '(paragraph-ignore-fill-prefix t)),
then it screws up.  I did not know that custom-set-variables
could have side effects.

>I suppose your old filladapt code still gets called
>somewhere in between.  You should try to get rid of
>it somehow, at least for testing this.

No way.  I think I confused you by even mentioning
that I had had an old version.  I really did fix
that.  (I also realize now that the reason I had a copy
in my private lisp directory is that filladapt is not part
of the regular Emacs package.)

Now it turns out that there is an issue about which
fill-region gets called.  With emacs -Q there is no
(require 'filladapt), so the fill-region is that from
fill.el.  However, the behaviours (right and wrong)
occur the same with either fill-region.

> In any case the `fill-region' call in `display-warning' should handle
> any prefix specified by `warning-fill-prefix' so this should not be ever
> ignored via `paragraph-ignore-fill-prefix'.  Just that I don't see any
> code where `paragraph-ignore-fill-prefix' might affect the behavior of
> `fill-region'.

I would like to think that fill-region is doing
nothing wrong.  OTOH, I have now verified that, with
emacs -Q, fill-region is still being called on the
messages with no \n; and, when there is a "In
<defun-name>:" line preceding, it is included in the
region and it does not get fouled up.  Speculation:
Under the right circumstances the colon in the defun
ID line somehow makes that a paragraph end.

I'll be interested to see if you can reproduce it now.
I remain willing to pursue other lines of inquiry if
you cannot.  The problem also exists in Emacs v22.

Regards,
  David V.