Re: Last steps for pretesting (font-lock-extend-region-function)

[Stefan Monnier]

> I find the elisp manual page which describes font-lock-keywords
> ("Search-based Fontification") all but inpenetrable.  It's reduced me to
> tears on several occasions.  I've just spent five minutes looking through
> it, trying to find the description of the form you suggested to me:

>      (".*\\\\\n.*"
>       (0 (progn (put-text-property (match-beginning 0) (match-end 0)
>                                    'font-lock-multiline t)
>                 nil)))
 
> .  I didn't find it.

Then the problem isn't font-lock-multiline.  Please don't mix up matters.
This form is the standard form (except you haven't noticed yet that the
FACENAME part of the font-lock-keywords is not a face symbol but an Elisp
expression whose value after exavluation should be a face).  It's:

 (MATCHER HIGHLIGHT ...)

where MATCHER is ".*\\\\\n.*" and HIGHLIGHT is of the form MATCH-HIGHLIGHT
which is of the form (SUBEXP FACENAME [OVERRIDE [LAXMATCH]]) where SUBEXP is
0 and FACENAME is (progn ...).

> With respect, Stefan, that's rubbish.  You need to know where and how to
> write the above Lisp form (the one starting (".*\\\\n.*" ...)) into
> f-l-keywords.  For me, that's perhaps half an hour to an hour of
> frustration trying to find the right place in "Search-based
> Fontification".

If you don't know how font-lock-keywords work, then learn that first before
trying to work on the font-lock support for cc-mode.  That'll save you a lot
more time.

> What about the run-time overhead of having an extra text property over
> the entire buffer?

It's not over the entire buffer.  And I haven't seen any evidence that it's
noticeable (especially compared the number of `face' properties or of the
`fontified' property added to the whole buffer).

> The f-l-e-r-f, as you envisage it (being called only from
> f-l-default-fontify-r) will also be called at virtually every buffer
> change, since almost every change causes a redisplay.

AFAIK, the case where a single change takes place between redisplay is the
case where there is no performance issue: it takes place at the user's
pace anyway.  The important case is when Emacs has more work to do in
response to a user's action, i.e. there's more work to be done until the
next redisplay.

>> Just take a deep breath, a couple of steps back, and try to look at the
>> example use of font-lock-multiline above as if it were very simple.

> I've tried, but I just don't have the intellectual capability to cope
> with such complexity without the greatest of difficulty.

The whole point is that you shouldn't have to.  It's like recursion (tho not
nearly as elegant, I must admit): have faith, don't try to simulate it all
in your head, and then it becomes simple.

> You obviously do.  Complicated nested structures make my eyes glaze over.
> There's a good chance I'm not the only Emacs hacker who has this sort
> of trouble.

Any Emacs hacker that needs to write complex syntax highlighting rules will
have to learn it anyway unless he prefers to circumvent font-lock and roll
his own instead.  Sounds like NIH to me.

> There's an assymetry between our positions: by threatening to remove the
> f-l-e-r-f hook (after-change version), you're trying to force me to use
> the f-l-m mechanism.

Yes.

> You're trying to impose your taste and judgement on me, and I resent this
> quite a lot.

I'm as close as it gets to a font-lock maintainer.  So from where I stand,
you're trying to impose your taste and judgment on me.

> I think the f-l-m text property is a revolting kludge, but
> I'm not trying to do away with it.  I'm happy enough for other hackers to
> use it, though I'd rather not have to maintain their code afterwards.

> Is there any chance we could bring this discussion to an end, now?  The
> things we've been talking about are important, but this discussion has
> taken up an enormous amount of my time, and probably yours too.  We've
> both got other urgent things to do.

3 ways this can end:
- Richard says that you're right.
- You convince me that the a-c-f code is better for some realistic case
  (e.g. yours).
- I remove the hook in a-c-f.

> My fix is simpler: to recognise that f-l-default-fontify-region performs
> two disparate tasks: (i) Extending the region to be fontified; (ii)
> Fontifying this region.  By extracting (ii) into a separate function,
> that function could be called directly from jit-lock-fontify-now,
> bypassing the redundant second massaging of the region.

It has to be in font-lock-default-fontify-buffer since ther'es no guarantee
this is only called from jit-lock-fontify-now.  So you're suggesting to add
a redundant call to the extend-region hook.  Maybe it's simpler but it's
ugly (because of the redundance, and because it adds a dependency between
jit-lock and font-lock).

> (i) It uses the same hook, font-lock-extend-region-function, as
> f-l-after-change-functions, setting the third parameter, OLD-LEN, to nil;

> (ii) It uses a distinct hook with a new name, say,
> font-lock-extend-chunk-function;

> (iii) It usurps the name f-l-e-r-f from its current use, abolishing the
> region extension mechanism in the f-l-after-change-f/j-l-after-change.

(iv) it reuses the name font-lock-extend-region-function and renames the
     a-c-f hook to font-lock-after-change-extend-region-function.

This will happen hopefully today.  At the same time
font-lock-after-change-extend-region-function will be moved from font-core
to font-lock (where it belongs) and its make-variable-buffer-local call will
be removed as well (use make-local-variable manually if you need it).


        Stefan