Re: font-lock-syntactic-keywords obsolet?

[Alan Mackenzie]

Hello, Dmitry.

On Sun, Jul 10, 2016 at 05:01:55AM +0300, Dmitry Gutov wrote:
> Hi Alan,

> Sorry for the late response.

No problems!

> On 06/30/2016 12:52 PM, Alan Mackenzie wrote:

> >> So now the raw strings are properly using limits? Does that mean there
> >> is a limit on the length of a raw string that CC Mode supports? (Testing
> >> indicates so).

> > There isn't any limit on the length of a raw string that I know about,
> > nor should there be.  If you've got a test which shows there is such a
> > limit, please tell me about it!

> Hmm, maybe not a limit, but long raw strings still aren't getting 
> handled right.

Yes.

> Example 1:

> - Apply the attached patch to xdisp.c, which should make most of the 
> code belong within the raw string literal.
> - Visit this file. Switch to c++-mode.
> - See the literal highlighted as expected.

> Press `M->', to get to the end of the buffer (that happens rather 
> slowly, esp. considering we're inside a string, and font-lock can get 
> this information quickly).

> The literal ends at )foo".

> - Modify the trailing "foo" piece: delete it, or replace with "bar", etc 
> => the literal still ends at the same line.

> I have to go back to the opener and fiddle with the delimiter there, for 
> it to finally notice that something is wrong.

> If the raw string is small, on the other hand, I don't see this problem.

> Example 2:

> - Visit this file. Switch to c++-mode.
> - See the literal highlighted as expected.
> - M->.

> Kill the closing delimiter and paste it a few lines below the opening 
> delimiter. See the new positions of the raw string recognized (or not, 
> I'm getting different results). But if they are recognized...

> - Call `undo' a few times, until the closing delimiter is back at its 
> original position. The literal is broken again.

There was some code in the mix designed to stop too much expansion of
a region when there were humongous macros.  (A few years back somebody
had complained about the speed in processing a ~5,000 line macro.)
Unfortunately, this code got caught up in raw string processing.  I hope
the following patch fixes it.  I know that the processing is currently
slow in such a large raw string.  It is probably possible to optimise
this.  Whether it is worthwhile is the question.



diff -r 2fcfc6e054b3 cc-mode.el
--- a/cc-mode.el        Sun Jul 03 17:54:20 2016 +0000
+++ b/cc-mode.el        Sun Jul 10 21:53:29 2016 +0000
@@ -906,14 +906,16 @@
   ;; before change function.
   (goto-char c-new-BEG)
   (c-beginning-of-macro)
-  (setq c-new-BEG (point))
+  (when (< (point) c-new-BEG)
+    (setq c-new-BEG (max (point) (c-determine-limit 500 c-new-BEG))))
 
   (goto-char c-new-END)
   (when (c-beginning-of-macro)
     (c-end-of-macro)
     (or (eobp) (forward-char)))         ; Over the terminating NL which may be 
marked
                                 ; with a c-cpp-delimiter category property
-  (setq c-new-END (point)))
+  (when (> (point) c-new-END)
+    (setq c-new-END (min (point) (c-determine-+ve-limit 500 c-new-END)))))
 
 (defun c-depropertize-new-text (beg end old-len)
   ;; Remove from the new text in (BEG END) any and all text properties which
@@ -941,15 +943,17 @@
   ;; Point is undefined on both entry and exit to this function.  The buffer
   ;; will have been widened on entry.
   ;;
+  ;; c-new-BEG has already been extended in `c-extend-region-for-CPP' so we
+  ;; don't need to repeat the exercise here.
+  ;;
   ;; This function is in the C/C++/ObjC value of 
`c-before-font-lock-functions'.
   (goto-char endd)
-  (if (c-beginning-of-macro)
-      (c-end-of-macro))
-  (setq c-new-END (max endd c-new-END (point)))
-  ;; Determine the region, (c-new-BEG c-new-END), which will get font
-  ;; locked.  This restricts the region should there be long macros.
-  (setq c-new-BEG (max c-new-BEG (c-determine-limit 500 begg))
-       c-new-END (min c-new-END (c-determine-+ve-limit 500 endd))))
+  (when (c-beginning-of-macro)
+    (c-end-of-macro)
+    ;; Determine the region, (c-new-BEG c-new-END), which will get font
+    ;; locked.  This restricts the region should there be long macros.
+    (setq c-new-END (min (max c-new-END (point))
+                        (c-determine-+ve-limit 500 c-new-END)))))
 
 (defun c-neutralize-CPP-line (beg end)
   ;; BEG and END bound a region, typically a preprocessor line.  Put a



> > The "limit" in my previous post was a bound supplied as an argument to
> > c-font-lock-declarators, which does what it says.

> I'm confused. If, as we discussed before, syntax properties are applied 
> in before/after-functions, why does c-font-lock-declarations need to be 
> concerned with scanning for raw string bounds?

The raw string bounds have nothing to do with c-font-lock-declarators.
It's just that that function takes a bound which was hardly ever needed,
since the function stops when it reaches something which signalled the
end of a sequence of declarators (for example, a semicolon).  Hence the
fact that the bound given was wrong didn't get noticed.  However, with
raw strings in the game, when (point-max) was the bound, the function
actually ended up fruitlessly scanning to (point-max) rather than to the
`limit' it should have been scanning to.

The limit to c-font-lock-declarators is now `(min limit (point-max))'.
That way, when the buffer is narrowed to less than `limit', there won't
be an out of bounds error, and when there are unterminated raw strings,
there won't be useless scanning past `limit' either.

I'm not sure if the above will help much, but I hope it does.

-- 
Alan Mackenzie (Nuremberg, Germany).