emacs-diffs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Emacs-diffs] /srv/bzr/emacs/trunk r101406: New syntax-propertize functi


From: Stefan Monnier
Subject: [Emacs-diffs] /srv/bzr/emacs/trunk r101406: New syntax-propertize functionality.
Date: Sat, 11 Sep 2010 01:13:42 +0200
User-agent: Bazaar (2.0.3)

------------------------------------------------------------
revno: 101406
committer: Stefan Monnier <address@hidden>
branch nick: trunk
timestamp: Sat 2010-09-11 01:13:42 +0200
message:
  New syntax-propertize functionality.
  * lisp/font-lock.el (font-lock-syntactic-keywords): Make obsolete.
  (font-lock-fontify-syntactic-keywords-region): Move handling of
  font-lock-syntactically-fontified to...
  (font-lock-default-fontify-region): ...here.
  Let syntax-propertize-function take precedence.
  (font-lock-fontify-syntactically-region): Cal syntax-propertize.
  
  * lisp/emacs-lisp/regexp-opt.el (regexp-opt-depth): Skip named groups.
  
  * lisp/emacs-lisp/syntax.el (syntax-propertize-function)
  (syntax-propertize-chunk-size, syntax-propertize--done)
  (syntax-propertize-extend-region-functions): New vars.
  (syntax-propertize-wholelines, syntax-propertize-multiline)
  (syntax-propertize--shift-groups, syntax-propertize-via-font-lock)
  (syntax-propertize): New functions.
  (syntax-propertize-rules): New macro.
  (syntax-ppss-flush-cache): Set syntax-propertize--done.
  (syntax-ppss): Call syntax-propertize.
  
  * lisp/progmodes/ada-mode.el (ada-set-syntax-table-properties)
  (ada-after-change-function, ada-initialize-syntax-table-properties)
  (ada-handle-syntax-table-properties): Only define when
  syntax-propertize is not available.
  (ada-mode): Use syntax-propertize-function.
  
  * lisp/progmodes/autoconf.el (autoconf-mode):
  Use syntax-propertize-function.
  (autoconf-font-lock-syntactic-keywords): Remove.
  
  * lisp/progmodes/cfengine.el (cfengine-mode):
  Use syntax-propertize-function.
  (cfengine-font-lock-syntactic-keywords): Remove.
  
  * lisp/progmodes/cperl-mode.el (cperl-mode): Use syntax-propertize-function.
  
  * lisp/progmodes/fortran.el (fortran-mode): Use syntax-propertize-function.
  (fortran--font-lock-syntactic-keywords): New var.
  (fortran-line-length): Update syntax-propertize-function and
  fortran--font-lock-syntactic-keywords.
  
  * lisp/progmodes/gud.el (gdb-script-syntax-propertize-function): New var;
  replaces gdb-script-font-lock-syntactic-keywords.
  (gdb-script-mode): Use it.
  
  * lisp/progmodes/js.el (js--regexp-literal): Define while compiling.
  (js-syntax-propertize-function): New var; replaces
  js-font-lock-syntactic-keywords.
  (js-mode): Use it.
  
  * lisp/progmodes/make-mode.el (makefile-syntax-propertize-function):
  New var; replaces makefile-font-lock-syntactic-keywords.
  (makefile-mode): Use it.
  (makefile-imake-mode): Adjust.
  
  * lisp/progmodes/mixal-mode.el (mixal-syntax-propertize-function): New var;
  replaces mixal-font-lock-syntactic-keywords.
  (mixal-mode): Use it.
  
  * lisp/progmodes/octave-mod.el (octave-syntax-propertize-sqs): New function
  to replace octave-font-lock-close-quotes.
  (octave-syntax-propertize-function): New function to replace
  octave-font-lock-syntactic-keywords.
  (octave-mode): Use it.
  
  * lisp/progmodes/perl-mode.el (perl-syntax-propertize-function): New fun to
  replace perl-font-lock-syntactic-keywords.
  (perl-syntax-propertize-special-constructs): New fun to replace
  perl-font-lock-special-syntactic-constructs.
  (perl-font-lock-syntactic-face-function): New fun.
  (perl-mode): Use it.
  
  * lisp/progmodes/python.el (python-syntax-propertize-function): New var to
  replace python-font-lock-syntactic-keywords.
  (python-mode): Use it.
  (python-quote-syntax): Simplify and adjust to new use.
  
  * lisp/progmodes/ruby-mode.el (ruby-here-doc-beg-re):
  Define while compiling.
  (ruby-here-doc-end-re, ruby-here-doc-beg-match)
  (ruby-font-lock-syntactic-keywords, ruby-comment-beg-syntax)
  (syntax-ppss, ruby-in-ppss-context-p, ruby-in-here-doc-p)
  (ruby-here-doc-find-end, ruby-here-doc-beg-syntax)
  (ruby-here-doc-end-syntax): Only define when
  syntax-propertize is not available.
  (ruby-syntax-propertize-function, ruby-syntax-propertize-heredoc):
  New functions.
  (ruby-in-ppss-context-p): Update to new syntax of heredocs.
  (electric-indent-chars): Silence bytecompiler.
  (ruby-mode): Use prog-mode, syntax-propertize-function, and
  electric-indent-chars.
  
  * lisp/progmodes/sh-script.el (sh-st-symbol): Remove.
  (sh-font-lock-close-heredoc, sh-font-lock-open-heredoc): Add eol arg.
  (sh-font-lock-flush-syntax-ppss-cache, sh-font-lock-here-doc): Remove.
  (sh-font-lock-quoted-subshell): Assume we've already matched $(.
  (sh-font-lock-paren): Set syntax-multiline.
  (sh-font-lock-syntactic-keywords): Remove.
  (sh-syntax-propertize-function): New function to replace it.
  (sh-mode): Use it.
  
  * lisp/progmodes/simula.el (simula-syntax-propertize-function): New var to
  replace simula-font-lock-syntactic-keywords.
  (simula-mode): Use it.
  
  * lisp/progmodes/tcl.el (tcl-syntax-propertize-function): New var to
  replace tcl-font-lock-syntactic-keywords.
  (tcl-mode): Use it.
  
  * lisp/progmodes/vhdl-mode.el (vhdl-mode): Use syntax-propertize-function
  if available.
  (vhdl-fontify-buffer): Adjust.
  
  * lisp/textmodes/bibtex.el (bibtex-mode): Use syntax-propertize-function.
  
  * lisp/textmodes/reftex.el (font-lock-syntactic-keywords): Don't declare
  since we don't use it.
  
  * lisp/textmodes/sgml-mode.el (sgml-syntax-propertize-function): New var to
  replace sgml-font-lock-syntactic-keywords.
  (sgml-mode): Use it.
  
  * lisp/textmodes/tex-mode.el (tex-common-initialization, doctex-mode):
  Use syntax-propertize-function.
  
  * lisp/textmodes/texinfo.el (texinfo-syntax-propertize-function): New fun
  to replace texinfo-font-lock-syntactic-keywords.
  (texinfo-mode): Use it.
  
  * test/indent/octave.m: Remove some `fixindent' not needed any more.
modified:
  etc/NEWS
  lisp/ChangeLog
  lisp/ansi-color.el
  lisp/emacs-lisp/regexp-opt.el
  lisp/emacs-lisp/syntax.el
  lisp/font-lock.el
  lisp/progmodes/ada-mode.el
  lisp/progmodes/autoconf.el
  lisp/progmodes/cfengine.el
  lisp/progmodes/cperl-mode.el
  lisp/progmodes/fortran.el
  lisp/progmodes/gud.el
  lisp/progmodes/js.el
  lisp/progmodes/make-mode.el
  lisp/progmodes/mixal-mode.el
  lisp/progmodes/octave-mod.el
  lisp/progmodes/perl-mode.el
  lisp/progmodes/python.el
  lisp/progmodes/ruby-mode.el
  lisp/progmodes/sh-script.el
  lisp/progmodes/simula.el
  lisp/progmodes/tcl.el
  lisp/progmodes/vhdl-mode.el
  lisp/textmodes/bibtex.el
  lisp/textmodes/reftex.el
  lisp/textmodes/sgml-mode.el
  lisp/textmodes/tex-mode.el
  lisp/textmodes/texinfo.el
  test/ChangeLog
  test/indent/octave.m
=== modified file 'etc/NEWS'
--- a/etc/NEWS  2010-09-10 17:51:48 +0000
+++ b/etc/NEWS  2010-09-10 23:13:42 +0000
@@ -466,6 +466,16 @@
 
 * Lisp changes in Emacs 24.1
 
+** New variable syntax-propertize-function to set syntax-table properties.
+Replaces font-lock-syntactic-keywords which are now obsolete.
+This allows syntax-table properties to be set independently from font-lock:
+just call syntax-propertize to make sure the text is propertized.
+Together with this new variable come a new hook
+syntax-propertize-extend-region-functions, as well as two helper functions:
+syntax-propertize-via-font-lock to reuse old font-lock-syntactic-keywords
+as-is; and syntax-propertize-rules which provides a new way to specify
+syntactic rules.
+
 ** New hook post-self-insert-hook run at the end of self-insert-command.
 
 ** Syntax tables support a new "comment style c" additionally to style b.

=== modified file 'lisp/ChangeLog'
--- a/lisp/ChangeLog    2010-09-10 11:20:51 +0000
+++ b/lisp/ChangeLog    2010-09-10 23:13:42 +0000
@@ -1,3 +1,133 @@
+2010-09-10  Stefan Monnier  <address@hidden>
+
+       * textmodes/texinfo.el (texinfo-syntax-propertize-function): New fun
+       to replace texinfo-font-lock-syntactic-keywords.
+       (texinfo-mode): Use it.
+
+       * textmodes/tex-mode.el (tex-common-initialization, doctex-mode):
+       Use syntax-propertize-function.
+
+       * textmodes/sgml-mode.el (sgml-syntax-propertize-function): New var to
+       replace sgml-font-lock-syntactic-keywords.
+       (sgml-mode): Use it.
+
+       * textmodes/reftex.el (font-lock-syntactic-keywords): Don't declare
+       since we don't use it.
+
+       * textmodes/bibtex.el (bibtex-mode): Use syntax-propertize-function.
+
+       * progmodes/vhdl-mode.el (vhdl-mode): Use syntax-propertize-function
+       if available.
+       (vhdl-fontify-buffer): Adjust.
+
+       * progmodes/tcl.el (tcl-syntax-propertize-function): New var to
+       replace tcl-font-lock-syntactic-keywords.
+       (tcl-mode): Use it.
+
+       * progmodes/simula.el (simula-syntax-propertize-function): New var to
+       replace simula-font-lock-syntactic-keywords.
+       (simula-mode): Use it.
+
+       * progmodes/sh-script.el (sh-st-symbol): Remove.
+       (sh-font-lock-close-heredoc, sh-font-lock-open-heredoc): Add eol arg.
+       (sh-font-lock-flush-syntax-ppss-cache, sh-font-lock-here-doc): Remove.
+       (sh-font-lock-quoted-subshell): Assume we've already matched $(.
+       (sh-font-lock-paren): Set syntax-multiline.
+       (sh-font-lock-syntactic-keywords): Remove.
+       (sh-syntax-propertize-function): New function to replace it.
+       (sh-mode): Use it.
+
+       * progmodes/ruby-mode.el (ruby-here-doc-beg-re):
+       Define while compiling.
+       (ruby-here-doc-end-re, ruby-here-doc-beg-match)
+       (ruby-font-lock-syntactic-keywords, ruby-comment-beg-syntax)
+       (syntax-ppss, ruby-in-ppss-context-p, ruby-in-here-doc-p)
+       (ruby-here-doc-find-end, ruby-here-doc-beg-syntax)
+       (ruby-here-doc-end-syntax): Only define when
+       syntax-propertize is not available.
+       (ruby-syntax-propertize-function, ruby-syntax-propertize-heredoc):
+       New functions.
+       (ruby-in-ppss-context-p): Update to new syntax of heredocs.
+       (electric-indent-chars): Silence bytecompiler.
+       (ruby-mode): Use prog-mode, syntax-propertize-function, and
+       electric-indent-chars.
+
+       * progmodes/python.el (python-syntax-propertize-function): New var to
+       replace python-font-lock-syntactic-keywords.
+       (python-mode): Use it.
+       (python-quote-syntax): Simplify and adjust to new use.
+
+       * progmodes/perl-mode.el (perl-syntax-propertize-function): New fun to
+       replace perl-font-lock-syntactic-keywords.
+       (perl-syntax-propertize-special-constructs): New fun to replace
+       perl-font-lock-special-syntactic-constructs.
+       (perl-font-lock-syntactic-face-function): New fun.
+       (perl-mode): Use it.
+
+       * progmodes/octave-mod.el (octave-syntax-propertize-sqs): New function
+       to replace octave-font-lock-close-quotes.
+       (octave-syntax-propertize-function): New function to replace
+       octave-font-lock-syntactic-keywords.
+       (octave-mode): Use it.
+
+       * progmodes/mixal-mode.el (mixal-syntax-propertize-function): New var;
+       replaces mixal-font-lock-syntactic-keywords.
+       (mixal-mode): Use it.
+
+       * progmodes/make-mode.el (makefile-syntax-propertize-function):
+       New var; replaces makefile-font-lock-syntactic-keywords.
+       (makefile-mode): Use it.
+       (makefile-imake-mode): Adjust.
+
+       * progmodes/js.el (js--regexp-literal): Define while compiling.
+       (js-syntax-propertize-function): New var; replaces
+       js-font-lock-syntactic-keywords.
+       (js-mode): Use it.
+
+       * progmodes/gud.el (gdb-script-syntax-propertize-function): New var;
+       replaces gdb-script-font-lock-syntactic-keywords.
+       (gdb-script-mode): Use it.
+
+       * progmodes/fortran.el (fortran-mode): Use syntax-propertize-function.
+       (fortran--font-lock-syntactic-keywords): New var.
+       (fortran-line-length): Update syntax-propertize-function and
+       fortran--font-lock-syntactic-keywords.
+
+       * progmodes/cperl-mode.el (cperl-mode): Use syntax-propertize-function.
+
+       * progmodes/cfengine.el (cfengine-mode):
+       Use syntax-propertize-function.
+       (cfengine-font-lock-syntactic-keywords): Remove.
+
+       * progmodes/autoconf.el (autoconf-mode):
+       Use syntax-propertize-function.
+       (autoconf-font-lock-syntactic-keywords): Remove.
+
+       * progmodes/ada-mode.el (ada-set-syntax-table-properties)
+       (ada-after-change-function, ada-initialize-syntax-table-properties)
+       (ada-handle-syntax-table-properties): Only define when
+       syntax-propertize is not available.
+       (ada-mode): Use syntax-propertize-function.
+
+       * font-lock.el (font-lock-syntactic-keywords): Make obsolete.
+       (font-lock-fontify-syntactic-keywords-region): Move handling of
+       font-lock-syntactically-fontified to...
+       (font-lock-default-fontify-region): ...here.
+       Let syntax-propertize-function take precedence.
+       (font-lock-fontify-syntactically-region): Cal syntax-propertize.
+
+       * emacs-lisp/syntax.el (syntax-propertize-function)
+       (syntax-propertize-chunk-size, syntax-propertize--done)
+       (syntax-propertize-extend-region-functions): New vars.
+       (syntax-propertize-wholelines, syntax-propertize-multiline)
+       (syntax-propertize--shift-groups, syntax-propertize-via-font-lock)
+       (syntax-propertize): New functions.
+       (syntax-propertize-rules): New macro.
+       (syntax-ppss-flush-cache): Set syntax-propertize--done.
+       (syntax-ppss): Call syntax-propertize.
+
+       * emacs-lisp/regexp-opt.el (regexp-opt-depth): Skip named groups.
+
 2010-09-10  Agust??n Mart??n  <address@hidden>
 
        * textmodes/ispell.el (ispell-init-process): Improve comments.
@@ -6,8 +136,8 @@
 
 2010-09-09  Michael Albinus  <address@hidden>
 
-       * net/tramp-cache.el (tramp-parse-connection-properties): Set
-       tramp-autoload cookie.
+       * net/tramp-cache.el (tramp-parse-connection-properties):
+       Set tramp-autoload cookie.
 
 2010-09-09  Glenn Morris  <address@hidden>
 

=== modified file 'lisp/ansi-color.el'
--- a/lisp/ansi-color.el        2010-06-01 02:34:49 +0000
+++ b/lisp/ansi-color.el        2010-09-10 23:13:42 +0000
@@ -244,9 +244,9 @@
   (when (boundp 'font-lock-syntactic-keywords)
     (remove-text-properties beg end '(syntax-table nil)))
   ;; instead of just using (remove-text-properties beg end '(face
-  ;; nil)), we find regions with a non-nil face test-property, skip
+  ;; nil)), we find regions with a non-nil face text-property, skip
   ;; positions with the ansi-color property set, and remove the
-  ;; remaining face test-properties.
+  ;; remaining face text-properties.
   (while (setq beg (text-property-not-all beg end 'face nil))
     (setq beg (or (text-property-not-all beg end 'ansi-color t) end))
     (when (get-text-property beg 'face)

=== modified file 'lisp/emacs-lisp/regexp-opt.el'
--- a/lisp/emacs-lisp/regexp-opt.el     2010-01-13 08:35:10 +0000
+++ b/lisp/emacs-lisp/regexp-opt.el     2010-09-10 23:13:42 +0000
@@ -120,7 +120,7 @@
     (string-match regexp "")
     ;; Count the number of open parentheses in REGEXP.
     (let ((count 0) start last)
-      (while (string-match "\\\\(\\(\\?:\\)?" regexp start)
+      (while (string-match "\\\\(\\(\\?[0-9]*:\\)?" regexp start)
        (setq start (match-end 0))            ; Start of next search.
        (when (and (not (match-beginning 1))
                   (subregexp-context-p regexp (match-beginning 0) last))

=== modified file 'lisp/emacs-lisp/syntax.el'
--- a/lisp/emacs-lisp/syntax.el 2010-09-04 22:51:39 +0000
+++ b/lisp/emacs-lisp/syntax.el 2010-09-10 23:13:42 +0000
@@ -34,7 +34,6 @@
 
 ;; - do something about the case where the syntax-table is changed.
 ;;   This typically happens with tex-mode and its `$' operator.
-;; - move font-lock-syntactic-keywords in here.  Then again, maybe not.
 ;; - new functions `syntax-state', ... to replace uses of parse-partial-state
 ;;   with something higher-level (similar to syntax-ppss-context).
 ;; - interaction with mmm-mode.
@@ -47,6 +46,249 @@
 
 (defvar font-lock-beginning-of-syntax-function)
 
+;;; Applying syntax-table properties where needed.
+
+(defvar syntax-propertize-function nil
+  ;; Rather than a -functions hook, this is a -function because it's easier
+  ;; to do a single scan than several scans: with multiple scans, one cannot
+  ;; assume that the text before point has been propertized, so syntax-ppss
+  ;; gives unreliable results (and stores them in its cache to boot, so we'd
+  ;; have to flush that cache between each function, and we couldn't use
+  ;; syntax-ppss-flush-cache since that would not only flush the cache but also
+  ;; reset syntax-propertize--done which should not be done in this case).
+  "Mode-specific function to apply the syntax-table properties.
+Called with 2 arguments: START and END.")
+
+(defvar syntax-propertize-chunk-size 500)
+
+(defvar syntax-propertize-extend-region-functions
+  '(syntax-propertize-wholelines)
+  "Special hook run just before proceeding to propertize a region.
+This is used to allow major modes to help `syntax-propertize' find safe buffer
+positions as beginning and end of the propertized region.  Its most common use
+is to solve the problem of /identification/ of multiline elements by providing
+a function that tries to find such elements and move the boundaries such that
+they do not fall in the middle of one.
+Each function is called with two arguments (START and END) and it should return
+either a cons (NEW-START . NEW-END) or nil if no adjustment should be made.
+These functions are run in turn repeatedly until they all return nil.
+Put first the functions more likely to cause a change and cheaper to compute.")
+;; Mark it as a special hook which doesn't use any global setting
+;; (i.e. doesn't obey the element t in the buffer-local value).
+(make-variable-buffer-local 'syntax-propertize-extend-region-functions)
+
+(defun syntax-propertize-wholelines (start end)
+  (goto-char start)
+  (cons (line-beginning-position)
+        (progn (goto-char end)
+               (if (bolp) (point) (line-beginning-position 2)))))
+
+(defun syntax-propertize-multiline (beg end)
+  "Let `syntax-propertize' pay attention to the syntax-multiline property."
+  (when (and (> beg (point-min))
+            (get-text-property (1- beg) 'syntax-multiline))
+    (setq beg (or (previous-single-property-change beg 'syntax-multiline)
+                 (point-min))))
+  ;;
+  (when (get-text-property end 'font-lock-multiline)
+    (setq end (or (text-property-any end (point-max)
+                                    'syntax-multiline nil)
+                 (point-max))))
+  (cons beg end))
+
+(defvar syntax-propertize--done -1
+  "Position upto which syntax-table properties have been set.")
+(make-variable-buffer-local 'syntax-propertize--done)
+
+(defun syntax-propertize--shift-groups (re n)
+  (replace-regexp-in-string
+   "\\\\(\\?\\([0-9]+\\):"
+   (lambda (s)
+     (replace-match
+      (number-to-string (+ n (string-to-number (match-string 1 s))))
+      t t s 1))
+   re t t))
+
+(defmacro syntax-propertize-rules (&rest rules)
+  "Make a function that applies RULES for use in `syntax-propertize-function'.
+The function will scan the buffer, applying the rules where they match.
+The buffer is scanned a single time, like \"lex\" would, rather than once
+per rule.
+
+Each rule has the form (REGEXP HIGHLIGHT1 ... HIGHLIGHTn), where REGEXP
+is an expression (evaluated at time of macro-expansion) that returns a regexp,
+and where HIGHLIGHTs have the form (NUMBER SYNTAX) which means to
+apply the property SYNTAX to the chars matched by the subgroup NUMBER
+of the regular expression, if NUMBER did match.
+SYNTAX is an expression that returns a value to apply as `syntax-table'
+property.  Some expressions are handled specially:
+- if SYNTAX is a string, then it is converted with `string-to-syntax';
+- if SYNTAX has the form (prog1 EXP . EXPS) then the value returned by EXP
+  will be applied to the buffer before running EXPS and if EXP is a string it
+  is also converted with `string-to-syntax'.
+The SYNTAX expression is responsible to save the `match-data' if needed
+for subsequent HIGHLIGHTs.
+Also SYNTAX is free to move point, in which case RULES may not be applied to
+some parts of the text or may be applied several times to other parts.
+
+Note: back-references in REGEXPs do not work."
+  (declare (debug (&rest (form &rest
+                               (numberp
+                                [&or stringp
+                                     ("prog1" [&or stringp def-form] def-body)
+                                     def-form])))))
+  (let* ((offset 0)
+         (branches '())
+         ;; We'd like to use a real DFA-based lexer, usually, but since Emacs
+         ;; doesn't have one yet, we fallback on building one large regexp
+         ;; and use groups to determine which branch of the regexp matched.
+         (re
+          (mapconcat
+           (lambda (rule)
+             (let ((re (eval (car rule))))
+               (when (and (assq 0 rule) (cdr rules))
+                 ;; If there's more than 1 rule, and the rule want to apply
+                 ;; highlight to match 0, create an extra group to be able to
+                 ;; tell when *this* match 0 has succeeded.
+                 (incf offset)
+                 (setq re (concat "\\(" re "\\)")))
+               (setq re (syntax-propertize--shift-groups re offset))
+               (let ((code '())
+                     (condition
+                      (cond
+                       ((assq 0 rule) (if (zerop offset) t
+                                        `(match-beginning ,offset)))
+                       ((null (cddr rule))
+                        `(match-beginning ,(+ offset (car (cadr rule)))))
+                       (t
+                        `(or ,@(mapcar
+                                (lambda (case)
+                                  `(match-beginning ,(+ offset (car case))))
+                                (cdr rule))))))
+                     (nocode t)
+                     (offset offset))
+                 ;; If some of the subgroup rules include Elisp code, then we
+                 ;; need to set the match-data so it's consistent with what the
+                 ;; code expects.  If not, then we can simply use shifted
+                 ;; offset in our own code.
+                 (unless (zerop offset)
+                   (dolist (case (cdr rule))
+                     (unless (stringp (cadr case))
+                       (setq nocode nil)))
+                   (unless nocode
+                     (push `(let ((md (match-data 'ints)))
+                              ;; Keep match 0 as is, but shift everything else.
+                              (setcdr (cdr md) (nthcdr ,(* (1+ offset) 2) md))
+                              (set-match-data md))
+                           code)
+                     (setq offset 0)))
+                 ;; Now construct the code for each subgroup rules.
+                 (dolist (case (cdr rule))
+                   (assert (null (cddr case)))
+                   (let* ((gn (+ offset (car case)))
+                          (action (nth 1 case))
+                          (thiscode
+                           (cond
+                            ((stringp action)
+                             `((put-text-property
+                                (match-beginning ,gn) (match-end ,gn)
+                                'syntax-table
+                                ',(string-to-syntax action))))
+                            ((eq (car-safe action) 'ignore)
+                             (cdr action))
+                            ((eq (car-safe action) 'prog1)
+                             (if (stringp (nth 1 action))
+                                 `((put-text-property
+                                    (match-beginning ,gn) (match-end ,gn)
+                                    'syntax-table
+                                    ',(string-to-syntax (nth 1 action)))
+                                   ,@(nthcdr 2 action))
+                               `((let ((mb (match-beginning ,gn))
+                                       (me (match-end ,gn))
+                                       (syntax ,(nth 1 action)))
+                                   (if syntax
+                                       (put-text-property
+                                        mb me 'syntax-table syntax))
+                                   ,@(nthcdr 2 action)))))
+                            (t
+                             `((let ((mb (match-beginning ,gn))
+                                     (me (match-end ,gn))
+                                     (syntax ,action))
+                                 (if syntax
+                                     (put-text-property
+                                      mb me 'syntax-table syntax))))))))
+                               
+                     (if (or (not (cddr rule)) (zerop gn))
+                         (setq code (nconc (nreverse thiscode) code))
+                       (push `(if (match-beginning ,gn)
+                                  ;; Try and generate clean code with no
+                                  ;; extraneous progn.
+                                  ,(if (null (cdr thiscode))
+                                       (car thiscode)
+                                     `(progn ,@thiscode)))
+                             code))))
+                 (push (cons condition (nreverse code))
+                       branches))
+               (incf offset (regexp-opt-depth re))
+               re))
+           rules
+           "\\|")))
+    `(lambda (start end)
+       (goto-char start)
+       (while (and (< (point) end)
+                   (re-search-forward ,re end t))
+         (cond ,@(nreverse branches))))))
+
+(defun syntax-propertize-via-font-lock (keywords)
+  "Propertize for syntax in START..END using font-lock syntax.
+KEYWORDS obeys the format used in `font-lock-syntactic-keywords'.
+The return value is a function suitable for `syntax-propertize-function'."
+  (lexical-let ((keywords keywords))
+    (lambda (start end)
+      (with-no-warnings
+        (let ((font-lock-syntactic-keywords keywords))
+          (font-lock-fontify-syntactic-keywords-region start end)
+          ;; In case it was eval'd/compiled.
+          (setq keywords font-lock-syntactic-keywords))))))
+
+(defun syntax-propertize (pos)
+  "Ensure that syntax-table properties are set upto POS."
+  (when (and syntax-propertize-function
+             (< syntax-propertize--done pos))
+    ;; (message "Needs to syntax-propertize from %s to %s"
+    ;;          syntax-propertize--done pos)
+    (set (make-local-variable 'parse-sexp-lookup-properties) t)
+    (save-excursion
+      (with-silent-modifications
+        (let* ((start (max syntax-propertize--done (point-min)))
+               (end (max pos
+                         (min (point-max)
+                              (+ start syntax-propertize-chunk-size))))
+               (funs syntax-propertize-extend-region-functions))
+          (while funs
+            (let ((new (funcall (pop funs) start end)))
+              (if (or (null new)
+                      (and (>= (car new) start) (<= (cdr new) end)))
+                  nil
+                (setq start (car new))
+                (setq end (cdr new))
+                ;; If there's been a change, we should go through the
+                ;; list again since this new position may
+                ;; warrant a different answer from one of the funs we've
+                ;; already seen.
+                (unless (eq funs
+                            (cdr syntax-propertize-extend-region-functions))
+                  (setq funs syntax-propertize-extend-region-functions)))))
+          ;; Move the limit before calling the function, so the function
+          ;; can use syntax-ppss.
+          (setq syntax-propertize--done end)
+          ;; (message "syntax-propertizing from %s to %s" start end)
+          (remove-text-properties start end
+                                  '(syntax-table nil syntax-multiline nil))
+          (funcall syntax-propertize-function start end))))))
+
+;;; Incrementally compute and memoize parser state.
+
 (defsubst syntax-ppss-depth (ppss)
   (nth 0 ppss))
 
@@ -92,6 +334,8 @@
 (defalias 'syntax-ppss-after-change-function 'syntax-ppss-flush-cache)
 (defun syntax-ppss-flush-cache (beg &rest ignored)
   "Flush the cache of `syntax-ppss' starting at position BEG."
+  ;; Set syntax-propertize to refontify anything past beg.
+  (setq syntax-propertize--done (min beg syntax-propertize--done))
   ;; Flush invalid cache entries.
   (while (and syntax-ppss-cache (> (caar syntax-ppss-cache) beg))
     (setq syntax-ppss-cache (cdr syntax-ppss-cache)))
@@ -128,6 +372,7 @@
 Point is at POS when this function returns."
   ;; Default values.
   (unless pos (setq pos (point)))
+  (syntax-propertize pos)
   ;;
   (let ((old-ppss (cdr syntax-ppss-last))
        (old-pos (car syntax-ppss-last))

=== modified file 'lisp/font-lock.el'
--- a/lisp/font-lock.el 2010-09-08 16:21:23 +0000
+++ b/lisp/font-lock.el 2010-09-10 23:13:42 +0000
@@ -544,6 +544,8 @@
  contexts will not be affected.
 
 This is normally set via `font-lock-defaults'.")
+(make-obsolete-variable 'font-lock-syntactic-keywords
+                        'syntax-propertize-function "24.1")
 
 (defvar font-lock-syntax-table nil
   "Non-nil means use this syntax table for fontifying.
@@ -1133,8 +1135,14 @@
           (setq beg font-lock-beg end font-lock-end))
         ;; Now do the fontification.
         (font-lock-unfontify-region beg end)
-        (when font-lock-syntactic-keywords
-          (font-lock-fontify-syntactic-keywords-region beg end))
+        (when (and font-lock-syntactic-keywords
+                   (null syntax-propertize-function))
+          ;; Ensure the beginning of the file is properly syntactic-fontified.
+          (let ((start beg))
+            (when (< font-lock-syntactically-fontified start)
+              (setq start (max font-lock-syntactically-fontified (point-min)))
+              (setq font-lock-syntactically-fontified end))
+            (font-lock-fontify-syntactic-keywords-region start end)))
         (unless font-lock-keywords-only
           (font-lock-fontify-syntactically-region beg end loudly))
         (font-lock-fontify-keywords-region beg end loudly)))))
@@ -1437,11 +1445,6 @@
     ;; We wouldn't go through so much trouble if we didn't intend to use those
     ;; properties, would we?
     (set (make-local-variable 'parse-sexp-lookup-properties) t))
-  ;; Ensure the beginning of the file is properly syntactic-fontified.
-  (when (and font-lock-syntactically-fontified
-            (< font-lock-syntactically-fontified start))
-    (setq start (max font-lock-syntactically-fontified (point-min)))
-    (setq font-lock-syntactically-fontified end))
   ;; If `font-lock-syntactic-keywords' is a symbol, get the real keywords.
   (when (symbolp font-lock-syntactic-keywords)
     (setq font-lock-syntactic-keywords (font-lock-eval-keywords
@@ -1487,6 +1490,7 @@
 (defun font-lock-fontify-syntactically-region (start end &optional loudly)
   "Put proper face on each string and comment between START and END.
 START should be at the beginning of a line."
+  (syntax-propertize end)  ; Apply any needed syntax-table properties.
   (let ((comment-end-regexp
         (or font-lock-comment-end-skip
             (regexp-quote

=== modified file 'lisp/progmodes/ada-mode.el'
--- a/lisp/progmodes/ada-mode.el        2010-09-08 16:21:23 +0000
+++ b/lisp/progmodes/ada-mode.el        2010-09-10 23:13:42 +0000
@@ -834,10 +834,7 @@
 ;;
 ;; On Emacs, this is done through the `syntax-table' text property.  The
 ;; corresponding action is applied automatically each time the buffer
-;; changes.  If `font-lock-mode' is enabled (the default) the action is
-;; set up by `font-lock-syntactic-keywords'.  Otherwise, we do it
-;; manually in `ada-after-change-function'.  The proper method is
-;; installed by `ada-handle-syntax-table-properties'.
+;; changes via syntax-propertize-function.
 ;;
 ;; on XEmacs, the `syntax-table' property does not exist and we have to use a
 ;; slow advice to `parse-partial-sexp' to do the same thing.
@@ -937,6 +934,12 @@
            (insert (caddar change))
            (setq change (cdr change)))))))
 
+(unless (eval-when-compile (fboundp 'syntax-propertize-via-font-lock))
+  ;; Before `syntax-propertize', we had to use font-lock to apply syntax-table
+  ;; properties, and in some cases we even had to do it manually (in
+  ;; `ada-after-change-function').  `ada-handle-syntax-table-properties'
+  ;; decides which method to use.
+
 (defun ada-set-syntax-table-properties ()
   "Assign `syntax-table' properties in accessible part of buffer.
 In particular, character constants are said to be strings, #...#
@@ -991,6 +994,8 @@
     ;; Take care of `syntax-table' properties manually.
     (ada-initialize-syntax-table-properties)))
 
+) ;;(not (fboundp 'syntax-propertize))
+
 ;;------------------------------------------------------------------
 ;;  Testing the grammatical context
 ;;------------------------------------------------------------------
@@ -1187,8 +1192,13 @@
        '(ada-font-lock-keywords
         nil t
         ((?\_ . "w") (?# . "."))
-        beginning-of-line
-        (font-lock-syntactic-keywords . ada-font-lock-syntactic-keywords)))
+        beginning-of-line))
+
+  (if (eval-when-compile (fboundp 'syntax-propertize-via-font-lock))
+      (set (make-local-variable 'syntax-propertize-function)
+           (syntax-propertize-via-font-lock ada-font-lock-syntactic-keywords))
+    (set (make-local-variable 'font-lock-syntactic-keywords)
+         ada-font-lock-syntactic-keywords))
 
   ;; Set up support for find-file.el.
   (set (make-local-variable 'ff-other-file-alist)
@@ -1331,7 +1341,8 @@
   ;;  Run this after the hook to give the users a chance to activate
   ;;  font-lock-mode
 
-  (unless (featurep 'xemacs)
+  (unless (or (eval-when-compile (fboundp 'syntax-propertize-via-font-lock))
+              (featurep 'xemacs))
     (ada-initialize-syntax-table-properties)
     (add-hook 'font-lock-mode-hook 'ada-handle-syntax-table-properties nil t))
 

=== modified file 'lisp/progmodes/autoconf.el'
--- a/lisp/progmodes/autoconf.el        2010-01-13 08:35:10 +0000
+++ b/lisp/progmodes/autoconf.el        2010-09-10 23:13:42 +0000
@@ -43,9 +43,6 @@
 (defvar autoconf-mode-hook nil
   "Hook run by `autoconf-mode'.")
 
-(defconst autoconf-font-lock-syntactic-keywords
-  '(("\\<dnl\\>" 0 '(11))))
-
 (defconst autoconf-definition-regexp
   "AC_\\(SUBST\\|DEFINE\\(_UNQUOTED\\)?\\)(\\[*\\(\\sw+\\)\\]*")
 
@@ -94,8 +91,8 @@
        "^[ \t]*A[CM]_\\(\\sw\\|\\s_\\)+")
   (set (make-local-variable 'comment-start) "dnl ")
   (set (make-local-variable 'comment-start-skip) "\\(?:\\<dnl\\|#\\) +")
-  (set (make-local-variable 'font-lock-syntactic-keywords)
-       autoconf-font-lock-syntactic-keywords)
+  (set (make-local-variable 'syntax-propertize-function)
+       (syntax-propertize-rules ("\\<dnl\\>" (0 "<"))))
   (set (make-local-variable 'font-lock-defaults)
        `(autoconf-font-lock-keywords nil nil (("_" . "w"))))
   (set (make-local-variable 'imenu-generic-expression)

=== modified file 'lisp/progmodes/cfengine.el'
--- a/lisp/progmodes/cfengine.el        2010-01-13 08:35:10 +0000
+++ b/lisp/progmodes/cfengine.el        2010-09-10 23:13:42 +0000
@@ -83,12 +83,6 @@
     ;; File, acl &c in group:   { token ... }
     ("{[ \t]*\\([^ \t\n]+\\)" 1 font-lock-constant-face)))
 
-(defconst cfengine-font-lock-syntactic-keywords
-  ;; In the main syntax-table, backslash is marked as a punctuation, because
-  ;; of its use in DOS-style directory separators.  Here we try to recognize
-  ;; the cases where backslash is used as an escape inside strings.
-  '(("\\(\\(?:\\\\\\)+\\)\"" 1 "\\")))
-
 (defvar cfengine-imenu-expression
   `((nil ,(concat "^[ \t]*" (eval-when-compile
                              (regexp-opt cfengine-actions t))
@@ -237,13 +231,15 @@
   (set (make-local-variable 'fill-paragraph-function)
        #'cfengine-fill-paragraph)
   (define-abbrev-table 'cfengine-mode-abbrev-table cfengine-mode-abbrevs)
-  ;; Fixme: Use `font-lock-syntactic-keywords' to set the args of
-  ;; functions in evaluated classes to string syntax, and then obey
-  ;; syntax properties.
   (setq font-lock-defaults
-       '(cfengine-font-lock-keywords nil nil nil beginning-of-line
-          (font-lock-syntactic-keywords
-           . cfengine-font-lock-syntactic-keywords)))
+       '(cfengine-font-lock-keywords nil nil nil beginning-of-line))
+  ;; Fixme: set the args of functions in evaluated classes to string
+  ;; syntax, and then obey syntax properties.
+  (set (make-local-variable 'syntax-propertize-function)
+       ;; In the main syntax-table, \ is marked as a punctuation, because
+       ;; of its use in DOS-style directory separators.  Here we try to
+       ;; recognize the cases where \ is used as an escape inside strings.
+       (syntax-propertize-rules ("\\(\\(?:\\\\\\)+\\)\"" (1 "\\"))))
   (setq imenu-generic-expression cfengine-imenu-expression)
   (set (make-local-variable 'beginning-of-defun-function)
        #'cfengine-beginning-of-defun)

=== modified file 'lisp/progmodes/cperl-mode.el'
--- a/lisp/progmodes/cperl-mode.el      2010-05-07 03:21:06 +0000
+++ b/lisp/progmodes/cperl-mode.el      2010-09-10 23:13:42 +0000
@@ -1840,7 +1840,13 @@
   (make-local-variable 'cperl-syntax-state)
   (setq cperl-syntax-state nil)                ; reset syntaxification cache
   (if cperl-use-syntax-table-text-property
-      (progn
+      (if (boundp 'syntax-propertize-function)
+          (progn
+            ;; Reset syntaxification cache.
+            (set (make-local-variable 'cperl-syntax-done-to) nil)
+            (set (make-local-variable 'syntax-propertize-function)
+                 (lambda (start end)
+                   (goto-char start) (cperl-fontify-syntaxically end))))
        (make-local-variable 'parse-sexp-lookup-properties)
        ;; Do not introduce variable if not needed, we check it!
        (set 'parse-sexp-lookup-properties t)

=== modified file 'lisp/progmodes/fortran.el'
--- a/lisp/progmodes/fortran.el 2010-09-08 16:21:23 +0000
+++ b/lisp/progmodes/fortran.el 2010-09-10 23:13:42 +0000
@@ -483,6 +483,7 @@
   "Maximum highlighting for Fortran mode.
 Consists of level 3 plus all other intrinsics not already highlighted.")
 
+(defvar fortran--font-lock-syntactic-keywords)
 ;; Comments are real pain in Fortran because there is no way to
 ;; represent the standard comment syntax in an Emacs syntax table.
 ;; (We can do so for F90-style).  Therefore an unmatched quote in a
@@ -887,9 +888,11 @@
           fortran-font-lock-keywords-3
           fortran-font-lock-keywords-4)
          nil t ((?/ . "$/") ("_$" . "w"))
-         fortran-beginning-of-subprogram
-         (font-lock-syntactic-keywords
-          . fortran-font-lock-syntactic-keywords)))
+         fortran-beginning-of-subprogram))
+  (set (make-local-variable 'fortran--font-lock-syntactic-keywords)
+       (fortran-make-syntax-propertize-function))
+  (set (make-local-variable 'syntax-propertize-function)
+       (syntax-propertize-via-font-lock fortran--font-lock-syntactic-keywords))
   (set (make-local-variable 'imenu-case-fold-search) t)
   (set (make-local-variable 'imenu-generic-expression)
        fortran-imenu-generic-expression)
@@ -917,10 +920,13 @@
               (when (eq major-mode 'fortran-mode)
                 (setq fortran-line-length nchars
                       fill-column fortran-line-length
-                      new (fortran-font-lock-syntactic-keywords))
+                      new (fortran-make-syntax-propertize-function))
                 ;; Refontify only if necessary.
-                (unless (equal new font-lock-syntactic-keywords)
-                  (setq font-lock-syntactic-keywords new)
+                (unless (equal new fortran--font-lock-syntactic-keywords)
+                  (setq fortran--font-lock-syntactic-keywords new)
+                  (setq syntax-propertize-function
+                        (syntax-propertize-via-font-lock new))
+                  (syntax-ppss-flush-cache (point-min))
                   (if font-lock-mode (font-lock-mode 1))))))
           (if global
               (buffer-list)

=== modified file 'lisp/progmodes/gud.el'
--- a/lisp/progmodes/gud.el     2010-07-28 10:08:16 +0000
+++ b/lisp/progmodes/gud.el     2010-09-10 23:13:42 +0000
@@ -3123,10 +3123,12 @@
     ("\\$\\(\\w+\\)" (1 font-lock-variable-name-face))
     ("^\\s-*\\(\\w\\(\\w\\|\\s_\\)*\\)" (1 font-lock-keyword-face))))
 
-(defvar gdb-script-font-lock-syntactic-keywords
-  '(("^document\\s-.*\\(\n\\)" (1 "< b"))
-    ("^end\\>"
-     (0 (unless (eq (match-beginning 0) (point-min))
+(defconst gdb-script-syntax-propertize-function
+  (syntax-propertize-rules
+   ("^document\\s-.*\\(\n\\)" (1 "< b"))
+   ("^end\\(\\>\\)"
+    (1 (ignore
+        (unless (eq (match-beginning 0) (point-min))
           ;; We change the \n in front, which is more difficult, but results
           ;; in better highlighting.  If the doc is empty, the single \n is
           ;; both the beginning and the end of the docstring, which can't be
@@ -3138,10 +3140,9 @@
                              'syntax-table (eval-when-compile
                                              (string-to-syntax "> b")))
           ;; Make sure that rehighlighting the previous line won't erase our
-          ;; syntax-table property.
+          ;; syntax-table property and that modifying `end' will.
           (put-text-property (1- (match-beginning 0)) (match-end 0)
-                             'font-lock-multiline t)
-          nil)))))
+                             'syntax-multiline t)))))))
 
 (defun gdb-script-font-lock-syntactic-face (state)
   (cond
@@ -3239,10 +3240,13 @@
        #'gdb-script-end-of-defun)
   (set (make-local-variable 'font-lock-defaults)
        '(gdb-script-font-lock-keywords nil nil ((?_ . "w")) nil
-        (font-lock-syntactic-keywords
-         . gdb-script-font-lock-syntactic-keywords)
         (font-lock-syntactic-face-function
-         . gdb-script-font-lock-syntactic-face))))
+         . gdb-script-font-lock-syntactic-face)))
+  ;; Recognize docstrings.
+  (set (make-local-variable 'syntax-propertize-function)
+       gdb-script-syntax-propertize-function)
+  (add-hook 'syntax-propertize-extend-region-functions
+            #'syntax-propertize-multiline 'append 'local))
 
 
 ;;; tooltips for GUD

=== modified file 'lisp/progmodes/js.el'
--- a/lisp/progmodes/js.el      2010-09-08 16:21:23 +0000
+++ b/lisp/progmodes/js.el      2010-09-10 23:13:42 +0000
@@ -1660,18 +1660,19 @@
 ;; XXX: Javascript can continue a regexp literal across lines so long
 ;; as the newline is escaped with \. Account for that in the regexp
 ;; below.
-(defconst js--regexp-literal
+(eval-and-compile
+  (defconst js--regexp-literal
   
"[=(,:]\\(?:\\s-\\|\n\\)*\\(/\\)\\(?:\\\\/\\|[^/*]\\)\\(?:\\\\/\\|[^/]\\)*\\(/\\)"
   "Regexp matching a JavaScript regular expression literal.
 Match groups 1 and 2 are the characters forming the beginning and
-end of the literal.")
-
-;; we want to match regular expressions only at the beginning of
-;; expressions
-(defconst js-font-lock-syntactic-keywords
-  `((,js--regexp-literal (1 "|") (2 "|")))
-  "Syntactic font lock keywords matching regexps in JavaScript.
-See `font-lock-keywords'.")
+end of the literal."))
+
+
+(defconst js-syntax-propertize-function
+  (syntax-propertize-rules
+   ;; We want to match regular expressions only at the beginning of
+   ;; expressions.
+   (js--regexp-literal (1 "\"") (2 "\""))))
 
 ;;; Indentation
 
@@ -3303,10 +3304,9 @@
 
   (set (make-local-variable 'open-paren-in-column-0-is-defun-start) nil)
   (set (make-local-variable 'font-lock-defaults)
-       (list js--font-lock-keywords
-            nil nil nil nil
-            '(font-lock-syntactic-keywords
-               . js-font-lock-syntactic-keywords)))
+       '(js--font-lock-keywords))
+  (set (make-local-variable 'syntax-propertize-function)
+       js-syntax-propertize-function)
 
   (set (make-local-variable 'parse-sexp-ignore-comments) t)
   (set (make-local-variable 'parse-sexp-lookup-properties) t)

=== modified file 'lisp/progmodes/make-mode.el'
--- a/lisp/progmodes/make-mode.el       2010-08-22 20:02:16 +0000
+++ b/lisp/progmodes/make-mode.el       2010-09-10 23:13:42 +0000
@@ -505,15 +505,16 @@
    cpp-font-lock-keywords))
 
 
-(defconst makefile-font-lock-syntactic-keywords
-  ;; From sh-script.el.
-  ;; A `#' begins a comment in sh when it is unquoted and at the beginning
-  ;; of a word.  In the shell, words are separated by metacharacters.
-  ;; The list of special chars is taken from the single-unix spec of the
-  ;; shell command language (under `quoting') but with `$' removed.
-  '(("[^|&;<>()`\\\"' \t\n]\\(#+\\)" 1 "_")
-    ;; Change the syntax of a quoted newline so that it does not end a comment.
-    ("\\\\\n" 0 ".")))
+(defconst makefile-syntax-propertize-function
+  (syntax-propertize-rules
+   ;; From sh-script.el.
+   ;; A `#' begins a comment in sh when it is unquoted and at the beginning
+   ;; of a word.  In the shell, words are separated by metacharacters.
+   ;; The list of special chars is taken from the single-unix spec of the
+   ;; shell command language (under `quoting') but with `$' removed.
+   ("[^|&;<>()`\\\"' \t\n]\\(#+\\)" (1 "_"))
+   ;; Change the syntax of a quoted newline so that it does not end a comment.
+   ("\\\\\n" (0 "."))))
 
 (defvar makefile-imenu-generic-expression
   `(("Dependencies" makefile-previous-dependency 1)
@@ -872,9 +873,9 @@
        '(makefile-font-lock-keywords
          nil nil
          ((?$ . "."))
-         backward-paragraph
-         (font-lock-syntactic-keywords
-          . makefile-font-lock-syntactic-keywords)))
+         backward-paragraph))
+  (set (make-local-variable 'syntax-propertize-function)
+       makefile-syntax-propertize-function)
 
   ;; Add-log.
   (set (make-local-variable 'add-log-current-defun-function)
@@ -943,15 +944,9 @@
 (define-derived-mode makefile-imake-mode makefile-mode "Imakefile"
   "An adapted `makefile-mode' that knows about imake."
   :syntax-table makefile-imake-mode-syntax-table
-  (let ((base `(makefile-imake-font-lock-keywords ,@(cdr font-lock-defaults)))
-       new)
-    ;; Remove `font-lock-syntactic-keywords' entry from font-lock-defaults.
-    (mapc (lambda (elt)
-           (unless (and (consp elt)
-                        (eq (car elt) 'font-lock-syntactic-keywords))
-             (setq new (cons elt new))))
-         base)
-    (setq font-lock-defaults (nreverse new))))
+  (set (make-local-variable 'syntax-propertize-function) nil)
+  (setq font-lock-defaults
+        `(makefile-imake-font-lock-keywords ,@(cdr font-lock-defaults))))
 
 
 

=== modified file 'lisp/progmodes/mixal-mode.el'
--- a/lisp/progmodes/mixal-mode.el      2010-03-14 21:15:02 +0000
+++ b/lisp/progmodes/mixal-mode.el      2010-09-10 23:13:42 +0000
@@ -89,7 +89,7 @@
 (defvar mixal-mode-syntax-table
   (let ((st (make-syntax-table)))
     ;; We need to do a bit more to make fontlocking for comments work.
-    ;; See mixal-font-lock-syntactic-keywords.
+    ;; See use of syntax-propertize-function.
     ;; (modify-syntax-entry ?* "<" st)
     (modify-syntax-entry ?\n ">" st)
     st)
@@ -1028,13 +1028,14 @@
 
 
 ;;; Font-locking:
-(defvar mixal-font-lock-syntactic-keywords
-  ;; Normal comments start with a * in column 0 and end at end of line.
-  '(("^\\*" (0 '(11)))                  ;(string-to-syntax "<") == '(11)
-    ;; Every line can end with a comment which is placed after the operand.
-    ;; I assume here that mnemonics without operands can not have a comment.
-    ("^[[:alnum:]]*[ \t]+[[:alnum:]]+[ \t]+[^ \n\t]+[ \t]*\\([ \t]\\)[^\n \t]"
-     (1 '(11)))))
+(defconst mixal-syntax-propertize-function
+  (syntax-propertize-rules
+   ;; Normal comments start with a * in column 0 and end at end of line.
+   ("^\\*" (0 "<"))
+   ;; Every line can end with a comment which is placed after the operand.
+   ;; I assume here that mnemonics without operands can not have a comment.
+   ("^[[:alnum:]]*[ \t]+[[:alnum:]]+[ \t]+[^ \n\t]+[ \t]*\\([ \t]\\)[^\n \t]"
+    (1 "<"))))
 
 (defvar mixal-font-lock-keywords
   `(("^\\([A-Z0-9a-z]+\\)"
@@ -1110,9 +1111,9 @@
   (set (make-local-variable 'comment-start) "*")
   (set (make-local-variable 'comment-start-skip) "^\\*[ \t]*")
   (set (make-local-variable 'font-lock-defaults)
-       `(mixal-font-lock-keywords nil nil nil nil
-         (font-lock-syntactic-keywords . ,mixal-font-lock-syntactic-keywords)
-         (parse-sexp-lookup-properties . t)))
+       `(mixal-font-lock-keywords))
+  (set (make-local-variable 'syntax-propertize-function)
+       mixal-syntax-propertize-function)
   ;; might add an indent function in the future
   ;;  (set (make-local-variable 'indent-line-function) 'mixal-indent-line)
   (set (make-local-variable 'compile-command) (concat "mixasm "

=== modified file 'lisp/progmodes/octave-mod.el'
--- a/lisp/progmodes/octave-mod.el      2010-09-08 16:21:23 +0000
+++ b/lisp/progmodes/octave-mod.el      2010-09-10 23:13:42 +0000
@@ -179,38 +179,28 @@
         '(3 font-lock-function-name-face nil t)))
   "Additional Octave expressions to highlight.")
 
-(defvar octave-font-lock-syntactic-keywords
+(defun octave-syntax-propertize-function (start end)
+  (goto-char start)
+  (octave-syntax-propertize-sqs end)
+  (funcall (syntax-propertize-rules
   ;; Try to distinguish the string-quotes from the transpose-quotes.
-  '(("[[({,; ]\\('\\)" (1 "\"'"))
-    (octave-font-lock-close-quotes)))
+            ("[[({,; ]\\('\\)"
+             (1 (prog1 "\"'" (octave-syntax-propertize-sqs end)))))
+           (point) end))
 
-(defun octave-font-lock-close-quotes (limit)
-  "Fix the syntax-table of the closing quotes of single-quote strings."
-  ;; Freely inspired from perl-font-lock-special-syntactic-constructs.
-  (let ((state (syntax-ppss)))
-    (while (< (point) limit)
-      (cond
-       ((eq (nth 3 state) ?\')
+(defun octave-syntax-propertize-sqs (end)
+  "Propertize the content/end of single-quote strings."
+  (when (eq (nth 3 (syntax-ppss)) ?\')
         ;; A '..' string.
-        (save-excursion
-          (when (re-search-forward "\\(?:\\=\\|[^']\\)\\(?:''\\)*\\('\\)[^']"
-                                   nil t)
-            (goto-char (1- (point)))
-            ;; Remove any syntax-table property we may have applied to
-            ;; some of the (doubled) single quotes within the string.
-            ;; Since these are the only chars on which we place properties,
-            ;; we take a shortcut and just remove all properties.
-            (remove-text-properties (1+ (nth 8 state)) (match-beginning 1)
-                                    '(syntax-table nil))
+    (when (re-search-forward
+           "\\(?:\\=\\|[^']\\)\\(?:''\\)*\\('\\)\\($\\|[^']\\)" end 'move)
+      (goto-char (match-beginning 2))
             (when (eq (char-before (match-beginning 1)) ?\\)
               ;; Backslash cannot escape a single quote.
               (put-text-property (1- (match-beginning 1)) (match-beginning 1)
                                  'syntax-table (string-to-syntax ".")))
             (put-text-property (match-beginning 1) (match-end 1)
-                               'syntax-table (string-to-syntax "\"'"))))))
-
-      (setq state (parse-partial-sexp (point) limit nil nil state
-                                     'syntax-table)))))
+                         'syntax-table (string-to-syntax "\"'")))))
 
 (defcustom inferior-octave-buffer "*Inferior Octave*"
   "Name of buffer for running an inferior Octave process."
@@ -684,9 +674,10 @@
   (set (make-local-variable 'normal-auto-fill-function) 'octave-auto-fill)
 
   (set (make-local-variable 'font-lock-defaults)
-       '(octave-font-lock-keywords nil nil nil nil
-         (font-lock-syntactic-keywords . octave-font-lock-syntactic-keywords)
-         (parse-sexp-lookup-properties . t)))
+       '(octave-font-lock-keywords))
+
+  (set (make-local-variable 'syntax-propertize-function)
+       #'octave-syntax-propertize-function)
 
   (set (make-local-variable 'imenu-generic-expression)
        octave-mode-imenu-generic-expression)

=== modified file 'lisp/progmodes/perl-mode.el'
--- a/lisp/progmodes/perl-mode.el       2010-01-13 08:35:10 +0000
+++ b/lisp/progmodes/perl-mode.el       2010-09-10 23:13:42 +0000
@@ -250,59 +250,76 @@
 ;; y /.../.../
 ;;
 ;; <file*glob>
-(defvar perl-font-lock-syntactic-keywords
-  ;; TODO: here-documents ("<<\\(\\sw\\|['\"]\\)")
-  `(;; Turn POD into b-style comments
-    ("^\\(=\\)\\sw" (1 "< b"))
-    ("^=cut[ \t]*\\(\n\\)" (1 "> b"))
-    ;; Catch ${ so that ${var} doesn't screw up indentation.
-    ;; This also catches $' to handle 'foo$', although it should really
-    ;; check that it occurs inside a '..' string.
-    ("\\(\\$\\)[{']" (1 ". p"))
-    ;; Handle funny names like $DB'stop.
-    ("\\$ ?{?^?[_a-zA-Z][_a-zA-Z0-9]*\\('\\)[_a-zA-Z]" (1 "_"))
-    ;; format statements
-    ("^[ \t]*format.*=[ \t]*\\(\n\\)" (1 '(7)))
-    ;; Funny things in `sub' arg-specs like `sub myfun ($)' or `sub ($)'.
-    ;; Be careful not to match "sub { (...) ... }".
-    
("\\<sub\\(?:[[:space:]]+[^{}[:punct:][:space:]]+\\)?[[:space:]]*(\\([^)]+\\))"
-     1 '(1))
-    ;; Regexp and funny quotes.  Distinguishing a / that starts a regexp
-    ;; match from the division operator is ...interesting.
-    ;; Basically, / is a regexp match if it's preceded by an infix operator
-    ;; (or some similar separator), or by one of the special keywords
-    ;; corresponding to builtin functions that can take their first arg
-    ;; without parentheses.  Of course, that presume we're looking at the
-    ;; *opening* slash.  We can afford to mis-match the closing ones
-    ;; here, because they will be re-treated separately later in
-    ;; perl-font-lock-special-syntactic-constructs.
-    (,(concat "\\(?:\\(?:\\(?:^\\|[^$@&%[:word:]]\\)"
-              (regexp-opt '("split" "if" "unless" "until" "while" "split"
-                            "grep" "map" "not" "or" "and"))
-              "\\)\\|[?:.,;=!~({[]\\|\\(^\\)\\)[ \t\n]*\\(/\\)")
-     (2 (if (and (match-end 1)
-                 (save-excursion
-                   (goto-char (match-end 1))
-                   ;; Not 100% correct since we haven't finished setting up
-                   ;; the syntax-table before point, but better than nothing.
-                   (forward-comment (- (point-max)))
-                   (put-text-property (point) (match-end 2)
-                                      'jit-lock-defer-multiline t)
-                   (not (memq (char-before)
-                              '(?? ?: ?. ?, ?\; ?= ?! ?~ ?\( ?\[)))))
-            nil ;; A division sign instead of a regexp-match.
-          '(7))))
-    ("\\(^\\|[?:.,;=!~({[ \t]\\)\\([msy]\\|q[qxrw]?\\|tr\\)\\>\\s-*\\([^])}> 
\n\t]\\)"
-     ;; Nasty cases:
-     ;; /foo/m  $a->m  $#m $m @m %m
-     ;; \s (appears often in regexps).
-     ;; -s file
-     (3 (if (assoc (char-after (match-beginning 3))
-                  perl-quote-like-pairs)
-           '(15) '(7))))
-    ;; Find and mark the end of funny quotes and format statements.
-    (perl-font-lock-special-syntactic-constructs)
-    ))
+(defun perl-syntax-propertize-function (start end)
+  (let ((case-fold-search nil))
+    (goto-char start)
+    (perl-syntax-propertize-special-constructs end)
+    ;; TODO: here-documents ("<<\\(\\sw\\|['\"]\\)")
+    (funcall
+     (syntax-propertize-rules
+      ;; Turn POD into b-style comments.  Place the cut rule first since it's
+      ;; more specific.
+      ("^=cut\\>.*\\(\n\\)" (1 "> b"))
+      ("^\\(=\\)\\sw" (1 "< b"))
+      ;; Catch ${ so that ${var} doesn't screw up indentation.
+      ;; This also catches $' to handle 'foo$', although it should really
+      ;; check that it occurs inside a '..' string.
+      ("\\(\\$\\)[{']" (1 ". p"))
+      ;; Handle funny names like $DB'stop.
+      ("\\$ ?{?^?[_a-zA-Z][_a-zA-Z0-9]*\\('\\)[_a-zA-Z]" (1 "_"))
+      ;; format statements
+      ("^[ \t]*format.*=[ \t]*\\(\n\\)"
+       (1 (prog1 "\"" (perl-syntax-propertize-special-constructs end))))
+      ;; Funny things in `sub' arg-specs like `sub myfun ($)' or `sub ($)'.
+      ;; Be careful not to match "sub { (...) ... }".
+      
("\\<sub\\(?:[[:space:]]+[^{}[:punct:][:space:]]+\\)?[[:space:]]*(\\([^)]+\\))"
+       (1 "."))
+      ;; Regexp and funny quotes.  Distinguishing a / that starts a regexp
+      ;; match from the division operator is ...interesting.
+      ;; Basically, / is a regexp match if it's preceded by an infix operator
+      ;; (or some similar separator), or by one of the special keywords
+      ;; corresponding to builtin functions that can take their first arg
+      ;; without parentheses.  Of course, that presume we're looking at the
+      ;; *opening* slash.  We can afford to mis-match the closing ones
+      ;; here, because they will be re-treated separately later in
+      ;; perl-font-lock-special-syntactic-constructs.
+      ((concat "\\(?:\\(?:^\\|[^$@&%[:word:]]\\)"
+               (regexp-opt '("split" "if" "unless" "until" "while" "split"
+                             "grep" "map" "not" "or" "and"))
+               "\\|[?:.,;=!~({[]\\|\\(^\\)\\)[ \t\n]*\\(/\\)")
+       (2 (ignore
+           (if (and (match-end 1)       ; / at BOL.
+                    (save-excursion
+                      (goto-char (match-end 1))
+                      (forward-comment (- (point-max)))
+                      (put-text-property (point) (match-end 2)
+                                         'syntax-multiline t)
+                      (not (memq (char-before)
+                                 '(?? ?: ?. ?, ?\; ?= ?! ?~ ?\( ?\[)))))
+               nil ;; A division sign instead of a regexp-match.
+             (put-text-property (match-beginning 2) (match-end 2)
+                                'syntax-table (string-to-syntax "\""))
+             (perl-syntax-propertize-special-constructs end)))))
+      ("\\(^\\|[?:.,;=!~({[ \t]\\)\\([msy]\\|q[qxrw]?\\|tr\\)\\>\\s-*\\([^])}> 
\n\t]\\)"
+       ;; Nasty cases:
+       ;; /foo/m  $a->m  $#m $m @m %m
+       ;; \s (appears often in regexps).
+       ;; -s file
+       ;; sub tr {...}
+       (3 (ignore
+           (if (save-excursion (goto-char (match-beginning 0))
+                               (forward-word -1)
+                               (looking-at-p "sub[ \t\n]"))
+               ;; This is defining a function.
+               nil
+             (put-text-property (match-beginning 3) (match-end 3)
+                                'syntax-table
+                                (if (assoc (char-after (match-beginning 3))
+                                           perl-quote-like-pairs)
+                                    (string-to-syntax "|")
+                                  (string-to-syntax "\"")))
+             (perl-syntax-propertize-special-constructs end))))))
+     (point) end)))
 
 (defvar perl-empty-syntax-table
   (let ((st (copy-syntax-table)))
@@ -321,95 +338,123 @@
       (modify-syntax-entry close ")" st))
     st))
 
-(defun perl-font-lock-special-syntactic-constructs (limit)
-  ;; We used to do all this in a font-lock-syntactic-face-function, which
-  ;; did not work correctly because sometimes some parts of the buffer are
-  ;; treated with font-lock-syntactic-keywords but not with
-  ;; font-lock-syntactic-face-function (mostly because of
-  ;; font-lock-syntactically-fontified).  That meant that some syntax-table
-  ;; properties were missing.  So now we do the parse-partial-sexp loop
-  ;; ourselves directly from font-lock-syntactic-keywords, so we're sure
-  ;; it's done when necessary.
+(defun perl-syntax-propertize-special-constructs (limit)
+  "Propertize special constructs like regexps and formats."
   (let ((state (syntax-ppss))
         char)
-    (while (< (point) limit)
-      (cond
-       ((or (null (setq char (nth 3 state)))
-            (and (characterp char) (eq (char-syntax (nth 3 state)) ?\")))
-        ;; Normal text, or comment, or docstring, or normal string.
-        nil)
-       ((eq (nth 3 state) ?\n)
-        ;; A `format' command.
-        (save-excursion
-          (when (and (re-search-forward "^\\s *\\.\\s *$" nil t)
-                     (not (eobp)))
-            (put-text-property (point) (1+ (point)) 'syntax-table '(7)))))
-       (t
-        ;; This is regexp like quote thingy.
-        (setq char (char-after (nth 8 state)))
-        (save-excursion
-          (let ((twoargs (save-excursion
-                           (goto-char (nth 8 state))
-                           (skip-syntax-backward " ")
-                           (skip-syntax-backward "w")
-                           (member (buffer-substring
-                                    (point) (progn (forward-word 1) (point)))
-                                   '("tr" "s" "y"))))
-                (close (cdr (assq char perl-quote-like-pairs)))
-                (pos (point))
-                (st (perl-quote-syntax-table char)))
-            (if (not close)
-                ;; The closing char is the same as the opening char.
-                (with-syntax-table st
-                  (parse-partial-sexp (point) (point-max)
-                                      nil nil state 'syntax-table)
-                  (when twoargs
-                    (parse-partial-sexp (point) (point-max)
-                                        nil nil state 'syntax-table)))
-              ;; The open/close chars are matched like () [] {} and <>.
-              (let ((parse-sexp-lookup-properties nil))
-                (condition-case err
-                    (progn
-                      (with-syntax-table st
-                        (goto-char (nth 8 state)) (forward-sexp 1))
-                      (when twoargs
-                        (save-excursion
-                          ;; Skip whitespace and make sure that font-lock will
-                          ;; refontify the second part in the proper context.
-                          (put-text-property
-                           (point) (progn (forward-comment (point-max)) 
(point))
-                           'font-lock-multiline t)
-                          ;;
-                          (unless
-                              (or (eobp)
-                                  (save-excursion
-                                    (with-syntax-table
-                                        (perl-quote-syntax-table (char-after))
-                                      (forward-sexp 1))
-                                    (put-text-property pos (line-end-position)
-                                                       
'jit-lock-defer-multiline t)
-                                    (looking-at "\\s-*\\sw*e")))
-                            (put-text-property (point) (1+ (point))
-                                               'syntax-table
-                                               (if (assoc (char-after)
-                                                          
perl-quote-like-pairs)
-                                                   '(15) '(7)))))))
-                  ;; The arg(s) is not terminated, so it extends until EOB.
-                  (scan-error (goto-char (point-max))))))
-            ;; Point is now right after the arg(s).
-            ;; Erase any syntactic marks within the quoted text.
-            (put-text-property pos (1- (point)) 'syntax-table nil)
-            (when (eq (char-before (1- (point))) ?$)
-              (put-text-property (- (point) 2) (1- (point))
-                                 'syntax-table '(1)))
-            (put-text-property (1- (point)) (point)
-                               'syntax-table (if close '(15) '(7)))))))
-
-      (setq state (parse-partial-sexp (point) limit nil nil state
-                                     'syntax-table))))
-  ;; Tell font-lock that this needs not further processing.
-  nil)
-
+    (cond
+     ((or (null (setq char (nth 3 state)))
+          (and (characterp char) (eq (char-syntax (nth 3 state)) ?\")))
+      ;; Normal text, or comment, or docstring, or normal string.
+      nil)
+     ((eq (nth 3 state) ?\n)
+      ;; A `format' command.
+      (when (re-search-forward "^\\s *\\.\\s *\n" limit 'move)
+        (put-text-property (1- (point)) (point)
+                           'syntax-table (string-to-syntax "\""))))
+     (t
+      ;; This is regexp like quote thingy.
+      (setq char (char-after (nth 8 state)))
+      (let ((twoargs (save-excursion
+                       (goto-char (nth 8 state))
+                       (skip-syntax-backward " ")
+                       (skip-syntax-backward "w")
+                       (member (buffer-substring
+                                (point) (progn (forward-word 1) (point)))
+                               '("tr" "s" "y"))))
+            (close (cdr (assq char perl-quote-like-pairs)))
+            (st (perl-quote-syntax-table char)))
+        (when (with-syntax-table st
+               (if close
+                   ;; For paired delimiters, Perl allows nesting them, but
+                   ;; since we treat them as strings, Emacs does not count
+                   ;; those delimiters in `state', so we don't know how deep
+                   ;; we are: we have to go back to the beginning of this
+                   ;; "string" and count from there.
+                   (condition-case nil
+                        (progn
+                         ;; Start after the first char since it doesn't have
+                         ;; paren-syntax (an alternative would be to let-bind
+                         ;; parse-sexp-lookup-properties).
+                         (goto-char (1+ (nth 8 state)))
+                         (up-list 1)
+                         t)
+                     (scan-error nil))
+                 (not (or (nth 8 (parse-partial-sexp
+                                  (point) limit nil nil state 'syntax-table))
+                          ;; If we have a self-paired opener and a twoargs
+                          ;; command, the form is s/../../ so we have to skip
+                          ;; a second time.
+                          ;; In the case of s{...}{...}, we only handle the
+                          ;; first part here and the next below.
+                          (when (and twoargs (not close))
+                            (nth 8 (parse-partial-sexp
+                                    (point) limit
+                                    nil nil state 'syntax-table)))))))
+         ;; Point is now right after the arg(s).
+         (when (eq (char-before (1- (point))) ?$)
+           (put-text-property (- (point) 2) (1- (point))
+                              'syntax-table '(1)))
+         (put-text-property (1- (point)) (point)
+                            'syntax-table
+                            (if close
+                                (string-to-syntax "|")
+                              (string-to-syntax "\"")))
+         ;; If we have two args with a non-self-paired starter (e.g.
+         ;; s{...}{...}) we're right after the first arg, so we still have to
+         ;; handle the second part.
+         (when (and twoargs close)
+            ;; Skip whitespace and make sure that font-lock will
+            ;; refontify the second part in the proper context.
+            (put-text-property
+             (point) (progn (forward-comment (point-max)) (point))
+            'syntax-multiline t)
+            ;;
+           (when (< (point) limit)
+              (put-text-property (point) (1+ (point))
+                                 'syntax-table
+                                 (if (assoc (char-after)
+                                            perl-quote-like-pairs)
+                                     ;; Put an `e' in the cdr to mark this
+                                     ;; char as "second arg starter".
+                                    (string-to-syntax "|e")
+                                  (string-to-syntax "\"e")))
+             (forward-char 1)
+             ;; Re-use perl-syntax-propertize-special-constructs to handle the
+             ;; second part (the first delimiter of second part can't be
+             ;; preceded by "s" or "tr" or "y", so it will not be considered
+             ;; as twoarg).
+             (perl-syntax-propertize-special-constructs limit)))))))))
+
+(defun perl-font-lock-syntactic-face-function (state)
+  (cond
+   ((and (nth 3 state)
+         (eq ?e (cdr-safe (get-text-property (nth 8 state) 'syntax-table)))
+         ;; This is a second-arg of s{..}{...} form; let's check if this second
+         ;; arg is executable code rather than a string.  For that, we need to
+         ;; look for an "e" after this second arg, so we have to hunt for the
+         ;; end of the arg.  Depending on whether the whole arg has already
+         ;; been syntax-propertized or not, the end-char will have different
+         ;; syntaxes, so let's ignore syntax-properties temporarily so we can
+         ;; pretend it has not been syntax-propertized yet.
+         (let* ((parse-sexp-lookup-properties nil)
+                (char (char-after (nth 8 state)))
+                (paired (assq char perl-quote-like-pairs)))
+           (with-syntax-table (perl-quote-syntax-table char)
+             (save-excursion
+               (if (not paired)
+                   (parse-partial-sexp (point) (point-max)
+                                       nil nil state 'syntax-table)
+                 (condition-case nil
+                     (progn
+                       (goto-char (1+ (nth 8 state)))
+                       (up-list 1))
+                   (scan-error (goto-char (point-max)))))
+               (put-text-property (nth 8 state) (point)
+                                  'jit-lock-defer-multiline t)
+               (looking-at "[ \t]*\\sw*e")))))
+    nil)
+   (t (funcall (default-value 'font-lock-syntactic-face-function) state))))
 
 (defcustom perl-indent-level 4
   "*Indentation of Perl statements with respect to containing block."
@@ -574,9 +619,12 @@
                              perl-font-lock-keywords-1
                              perl-font-lock-keywords-2)
                             nil nil ((?\_ . "w")) nil
-                            (font-lock-syntactic-keywords
-                             . perl-font-lock-syntactic-keywords)
-                            (parse-sexp-lookup-properties . t)))
+                             (font-lock-syntactic-face-function
+                              . perl-font-lock-syntactic-face-function)))
+  (set (make-local-variable 'syntax-propertize-function)
+       #'perl-syntax-propertize-function)
+  (add-hook 'syntax-propertize-extend-region-functions
+            #'syntax-propertize-multiline 'append 'local)
   ;; Tell imenu how to handle Perl.
   (set (make-local-variable 'imenu-generic-expression)
        perl-imenu-generic-expression)

=== modified file 'lisp/progmodes/python.el'
--- a/lisp/progmodes/python.el  2010-09-01 07:13:21 +0000
+++ b/lisp/progmodes/python.el  2010-09-10 23:13:42 +0000
@@ -166,29 +166,32 @@
           symbol-end)
      . font-lock-builtin-face)))
 
-(defconst python-font-lock-syntactic-keywords
+(defconst python-syntax-propertize-function
   ;; Make outer chars of matching triple-quote sequences into generic
   ;; string delimiters.  Fixme: Is there a better way?
   ;; First avoid a sequence preceded by an odd number of backslashes.
-  `((,(rx (not (any ?\\))
-         ?\\ (* (and ?\\ ?\\))
-         (group (syntax string-quote))
-         (backref 1)
-         (group (backref 1)))
-     (2 ,(string-to-syntax "\"")))     ; dummy
-    (,(rx (group (optional (any "uUrR"))) ; prefix gets syntax property
-         (optional (any "rR"))           ; possible second prefix
-         (group (syntax string-quote))   ; maybe gets property
-         (backref 2)                     ; per first quote
-         (group (backref 2)))            ; maybe gets property
-     (1 (python-quote-syntax 1))
-     (2 (python-quote-syntax 2))
-     (3 (python-quote-syntax 3)))
-    ;; This doesn't really help.
-;;;     (,(rx (and ?\\ (group ?\n))) (1 " "))
-    ))
+  (syntax-propertize-rules
+   (;; (rx (not (any ?\\))
+    ;;     ?\\ (* (and ?\\ ?\\))
+    ;;     (group (syntax string-quote))
+    ;;     (backref 1)
+    ;;     (group (backref 1)))
+    ;; ?Backrefs don't work in syntax-propertize-rules!
+    "[^\\]\\\\\\(\\\\\\\\\\)*\\(?:''\\('\\)\\|\"\"\\(?2:\"\\)\\)"
+    (2 "\""))                           ; dummy
+   (;; (rx (optional (group (any "uUrR"))) ; prefix gets syntax property
+    ;;     (optional (any "rR"))           ; possible second prefix
+    ;;     (group (syntax string-quote))   ; maybe gets property
+    ;;     (backref 2)                     ; per first quote
+    ;;     (group (backref 2)))            ; maybe gets property
+    ;; ?Backrefs don't work in syntax-propertize-rules!
+    "\\([RUru]\\)?[Rr]?\\(?:\\('\\)'\\('\\)\\|\\(?2:\"\\)\"\\(?3:\"\\)\\)"
+    (3 (ignore (python-quote-syntax))))
+   ;; This doesn't really help.
+   ;;((rx (and ?\\ (group ?\n))) (1 " "))
+   ))
 
-(defun python-quote-syntax (n)
+(defun python-quote-syntax ()
   "Put `syntax-table' property correctly on triple quote.
 Used for syntactic keywords.  N is the match number (1, 2 or 3)."
   ;; Given a triple quote, we have to check the context to know
@@ -206,28 +209,25 @@
   ;; x '"""' x """ \"""" x
   (save-excursion
     (goto-char (match-beginning 0))
-    (cond
-     ;; Consider property for the last char if in a fenced string.
-     ((= n 3)
-      (let* ((font-lock-syntactic-keywords nil)
-            (syntax (syntax-ppss)))
-       (when (eq t (nth 3 syntax))     ; after unclosed fence
-         (goto-char (nth 8 syntax))    ; fence position
-         (skip-chars-forward "uUrR")   ; skip any prefix
-         ;; Is it a matching sequence?
-         (if (eq (char-after) (char-after (match-beginning 2)))
-             (eval-when-compile (string-to-syntax "|"))))))
-     ;; Consider property for initial char, accounting for prefixes.
-     ((or (and (= n 2)                 ; leading quote (not prefix)
-              (= (match-beginning 1) (match-end 1))) ; prefix is null
-         (and (= n 1)                  ; prefix
-              (/= (match-beginning 1) (match-end 1)))) ; non-empty
-      (let ((font-lock-syntactic-keywords nil))
-       (unless (eq 'string (syntax-ppss-context (syntax-ppss)))
-         (eval-when-compile (string-to-syntax "|")))))
-     ;; Otherwise (we're in a non-matching string) the property is
-     ;; nil, which is OK.
-     )))
+    (let ((syntax (save-match-data (syntax-ppss))))
+      (cond
+       ((eq t (nth 3 syntax))           ; after unclosed fence
+        ;; Consider property for the last char if in a fenced string.
+        (goto-char (nth 8 syntax))     ; fence position
+        (skip-chars-forward "uUrR")    ; skip any prefix
+        ;; Is it a matching sequence?
+        (if (eq (char-after) (char-after (match-beginning 2)))
+            (put-text-property (match-beginning 3) (match-end 3)
+                               'syntax-table (string-to-syntax "|"))))
+       ((match-end 1)
+        ;; Consider property for initial char, accounting for prefixes.
+        (put-text-property (match-beginning 1) (match-end 1)
+                           'syntax-table (string-to-syntax "|")))
+       (t
+        ;; Consider property for initial char, accounting for prefixes.
+        (put-text-property (match-beginning 2) (match-end 2)
+                           'syntax-table (string-to-syntax "|"))))
+      )))
 
 ;; This isn't currently in `font-lock-defaults' as probably not worth
 ;; it -- we basically only mess with a few normally-symbol characters.
@@ -2495,12 +2495,12 @@
   :group 'python
   (set (make-local-variable 'font-lock-defaults)
        '(python-font-lock-keywords nil nil nil nil
-                                  (font-lock-syntactic-keywords
-                                   . python-font-lock-syntactic-keywords)
-                                  ;; This probably isn't worth it.
-                                  ;; (font-lock-syntactic-face-function
-                                  ;;  . 
python-font-lock-syntactic-face-function)
-                                  ))
+         ;; This probably isn't worth it.
+         ;; (font-lock-syntactic-face-function
+         ;;  . python-font-lock-syntactic-face-function)
+         ))
+  (set (make-local-variable 'syntax-propertize-function)
+       python-syntax-propertize-function)
   (set (make-local-variable 'parse-sexp-lookup-properties) t)
   (set (make-local-variable 'parse-sexp-ignore-comments) t)
   (set (make-local-variable 'comment-start) "# ")

=== modified file 'lisp/progmodes/ruby-mode.el'
--- a/lisp/progmodes/ruby-mode.el       2010-08-24 20:29:44 +0000
+++ b/lisp/progmodes/ruby-mode.el       2010-09-10 23:13:42 +0000
@@ -100,17 +100,10 @@
 
 (defconst ruby-block-end-re "\\<end\\>")
 
-(defconst ruby-here-doc-beg-re
+(eval-and-compile
+  (defconst ruby-here-doc-beg-re
   
"\\(<\\)<\\(-\\)?\\(\\([a-zA-Z0-9_]+\\)\\|[\"]\\([^\"]+\\)[\"]\\|[']\\([^']+\\)[']\\)"
-  "Regexp to match the beginning of a heredoc.")
-
-(defconst ruby-here-doc-end-re
-  "^\\([ \t]+\\)?\\(.*\\)\\(.\\)$"
-  "Regexp to match the end of heredocs.
-
-This will actually match any line with one or more characters.
-It's useful in that it divides up the match string so that
-`ruby-here-doc-beg-match' can search for the beginning of the heredoc.")
+    "Regexp to match the beginning of a heredoc."))
 
 (defun ruby-here-doc-end-match ()
   "Return a regexp to find the end of a heredoc.
@@ -123,18 +116,6 @@
                (match-string 5)
                (match-string 6)))))
 
-(defun ruby-here-doc-beg-match ()
-  "Return a regexp to find the beginning of a heredoc.
-
-This should only be called after matching against `ruby-here-doc-end-re'."
-  (let ((contents (regexp-quote (concat (match-string 2) (match-string 3)))))
-    (concat "<<"
-            (let ((match (match-string 1)))
-              (if (and match (> (length match) 0))
-                  (concat "\\(?:-\\([\"']?\\)\\|\\([\"']\\)" (match-string 1) 
"\\)"
-                          contents "\\b\\(\\1\\|\\2\\)")
-                (concat "-?\\([\"']\\|\\)" contents "\\b\\1"))))))
-
 (defconst ruby-delimiter
   (concat "[?$/%(){}#\"'`.:]\\|<<\\|\\[\\|\\]\\|\\<\\("
           ruby-block-beg-re
@@ -362,7 +343,7 @@
     (back-to-indentation)
     (current-column)))
 
-(defun ruby-indent-line (&optional flag)
+(defun ruby-indent-line (&optional ignored)
   "Correct the indentation of the current Ruby line."
   (interactive)
   (ruby-indent-to (ruby-calculate-indent)))
@@ -405,8 +386,7 @@
   "TODO: document."
   (save-excursion
     (store-match-data nil)
-    (let ((space (skip-chars-backward " \t"))
-          (start (point)))
+    (let ((space (skip-chars-backward " \t")))
       (cond
        ((bolp) t)
        ((progn
@@ -700,7 +680,7 @@
     (beginning-of-line)
     (let ((ruby-indent-point (point))
           (case-fold-search nil)
-          state bol eol begin op-end
+          state eol begin op-end
           (paren (progn (skip-syntax-forward " ")
                         (and (char-after) (matching-paren (char-after)))))
           (indent 0))
@@ -780,7 +760,6 @@
               (if (re-search-forward "^\\s *#" end t)
                   (beginning-of-line)
                 (setq done t))))
-          (setq bol (point))
           (end-of-line)
           ;; skip the comment at the end
           (skip-chars-backward " \t")
@@ -1037,10 +1016,8 @@
   (ruby-beginning-of-defun)
   (re-search-backward "^\n" (- (point) 1) t))
 
-(defun ruby-indent-exp (&optional shutup-p)
-  "Indent each line in the balanced expression following the point.
-If a prefix arg is given or SHUTUP-P is non-nil, no errors
-are signalled if a balanced expression isn't found."
+(defun ruby-indent-exp (&optional ignored)
+  "Indent each line in the balanced expression following the point."
   (interactive "*P")
   (let ((here (point-marker)) start top column (nest t))
     (set-marker-insertion-type here t)
@@ -1133,58 +1110,208 @@
               (if mlist (concat mlist mname) mname)
             mlist)))))
 
-(defconst ruby-font-lock-syntactic-keywords
-  `(;; #{ }, #$hoge, address@hidden are not comments
-    ("\\(#\\)address@hidden" 1 (1 . nil))
-    ;; the last $', $", $` in the respective string is not variable
-    ;; the last ?', ?", ?` in the respective string is not ascii code
-    ("\\(^\\|[\[ 
\t\n<+\(,=]\\)\\(['\"`]\\)\\(\\\\.\\|\\2\\|[^'\"`\n\\\\]\\)*?\\\\?[?$]\\(\\2\\)"
-     (2 (7 . nil))
-     (4 (7 . nil)))
-    ;; $' $" $` .... are variables
-    ;; ?' ?" ?` are ascii codes
-    ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" 3 (1 . nil))
-    ;; regexps
-    ("\\(^\\|[=(,~?:;<>]\\|\\(^\\|\\s 
\\)\\(if\\|elsif\\|unless\\|while\\|until\\|when\\|and\\|or\\|&&\\|||\\)\\|g?sub!?\\|scan\\|split!?\\)\\s
 *\\(/\\)[^/\n\\\\]*\\(\\\\.[^/\n\\\\]*\\)*\\(/\\)"
-     (4 (7 . ?/))
-     (6 (7 . ?/)))
-    ("^=en\\(d\\)\\_>" 1 "!")
-    ("^\\(=\\)begin\\_>" 1 (ruby-comment-beg-syntax))
-    ;; Currently, the following case is highlighted incorrectly:
-    ;;
-    ;;   <<FOO
-    ;;   FOO
-    ;;   <<BAR
-    ;;   <<BAZ
-    ;;   BAZ
-    ;;   BAR
-    ;;
-    ;; This is because all here-doc beginnings are highlighted before any 
endings,
-    ;; so although <<BAR is properly marked as a beginning, when we get to 
<<BAZ
-    ;; it thinks <<BAR is part of a string so it's marked as well.
-    ;;
-    ;; This may be fixable by modifying ruby-in-here-doc-p to use
-    ;; ruby-in-non-here-doc-string-p rather than syntax-ppss-context,
-    ;; but I don't want to try that until we've got unit tests set up
-    ;; to make sure I don't break anything else.
-    (,(concat ruby-here-doc-beg-re ".*\\(\n\\)")
-     ,(+ 1 (regexp-opt-depth ruby-here-doc-beg-re))
-     (ruby-here-doc-beg-syntax))
-    (,ruby-here-doc-end-re 3 (ruby-here-doc-end-syntax)))
-  "Syntactic keywords for Ruby mode.  See `font-lock-syntactic-keywords'.")
-
-(defun ruby-comment-beg-syntax ()
-  "Return the syntax cell for a the first character of a =begin.
+(if (eval-when-compile (fboundp #'syntax-propertize-rules))
+    ;; New code that works independently from font-lock.
+    (progn
+      (defun ruby-syntax-propertize-function (start end)
+        "Syntactic keywords for Ruby mode.  See `syntax-propertize-function'."
+        (goto-char start)
+        (ruby-syntax-propertize-heredoc end)
+        (funcall
+         (syntax-propertize-rules
+          ;; #{ }, #$hoge, address@hidden are not comments
+          ("\\(#\\)address@hidden" (1 "."))
+          ;; the last $', $", $` in the respective string is not variable
+          ;; the last ?', ?", ?` in the respective string is not ascii code
+          ("\\(^\\|[\[ 
\t\n<+\(,=]\\)\\(['\"`]\\)\\(\\\\.\\|\\2\\|[^'\"`\n\\\\]\\)*?\\\\?[?$]\\(\\2\\)"
+           (2 "\"")
+           (4 "\""))
+          ;; $' $" $` .... are variables
+          ;; ?' ?" ?` are ascii codes
+          ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" (3 "."))
+          ;; regexps
+          ("\\(^\\|[=(,~?:;<>]\\|\\(^\\|\\s 
\\)\\(if\\|elsif\\|unless\\|while\\|until\\|when\\|and\\|or\\|&&\\|||\\)\\|g?sub!?\\|scan\\|split!?\\)\\s
 *\\(/\\)[^/\n\\\\]*\\(\\\\.[^/\n\\\\]*\\)*\\(/\\)"
+           (4 "\"/")
+           (6 "\"/"))
+          ("^=en\\(d\\)\\_>" (1 "!"))
+          ("^\\(=\\)begin\\_>" (1 "!"))
+          ;; Handle here documents.
+          ((concat ruby-here-doc-beg-re ".*\\(\n\\)")
+           (7 (prog1 "\"" (ruby-syntax-propertize-heredoc end)))))
+         (point) end))
+
+      (defun ruby-syntax-propertize-heredoc (limit)
+        (let ((ppss (syntax-ppss))
+              (res '()))
+          (when (eq ?\n (nth 3 ppss))
+            (save-excursion
+              (goto-char (nth 8 ppss))
+              (beginning-of-line)
+              (while (re-search-forward ruby-here-doc-beg-re
+                                        (line-end-position) t)
+                (push (concat (ruby-here-doc-end-match) "\n") res)))
+            (let ((start (point)))
+              ;; With multiple openers on the same line, we don't know in which
+              ;; part `start' is, so we have to go back to the beginning.
+              (when (cdr res)
+                (goto-char (nth 8 ppss))
+                (setq res (nreverse res)))
+              (while (and res (re-search-forward (pop res) limit 'move))
+                (if (null res)
+                    (put-text-property (1- (point)) (point)
+                                       'syntax-table (string-to-syntax "\""))))
+              ;; Make extra sure we don't move back, lest we could fall into an
+              ;; inf-loop.
+              (if (< (point) start) (goto-char start))))))
+      )
+      
+  ;; For Emacsen where syntax-propertize-rules is not (yet) available,
+  ;; fallback on the old font-lock-syntactic-keywords stuff.
+
+  (defconst ruby-here-doc-end-re
+    "^\\([ \t]+\\)?\\(.*\\)\\(\n\\)"
+    "Regexp to match the end of heredocs.
+
+This will actually match any line with one or more characters.
+It's useful in that it divides up the match string so that
+`ruby-here-doc-beg-match' can search for the beginning of the heredoc.")
+
+  (defun ruby-here-doc-beg-match ()
+    "Return a regexp to find the beginning of a heredoc.
+
+This should only be called after matching against `ruby-here-doc-end-re'."
+    (let ((contents (regexp-quote (match-string 2))))
+      (concat "<<"
+              (let ((match (match-string 1)))
+                (if (and match (> (length match) 0))
+                    (concat "\\(?:-\\([\"']?\\)\\|\\([\"']\\)" match "\\)"
+                            contents "\\b\\(\\1\\|\\2\\)")
+                  (concat "-?\\([\"']\\|\\)" contents "\\b\\1"))))))
+
+  (defconst ruby-font-lock-syntactic-keywords
+    `( ;; #{ }, #$hoge, address@hidden are not comments
+      ("\\(#\\)address@hidden" 1 (1 . nil))
+      ;; the last $', $", $` in the respective string is not variable
+      ;; the last ?', ?", ?` in the respective string is not ascii code
+      ("\\(^\\|[\[ 
\t\n<+\(,=]\\)\\(['\"`]\\)\\(\\\\.\\|\\2\\|[^'\"`\n\\\\]\\)*?\\\\?[?$]\\(\\2\\)"
+       (2 (7 . nil))
+       (4 (7 . nil)))
+      ;; $' $" $` .... are variables
+      ;; ?' ?" ?` are ascii codes
+      ("\\(^\\|[^\\\\]\\)\\(\\\\\\\\\\)*[?$]\\([#\"'`]\\)" 3 (1 . nil))
+      ;; regexps
+      ("\\(^\\|[=(,~?:;<>]\\|\\(^\\|\\s 
\\)\\(if\\|elsif\\|unless\\|while\\|until\\|when\\|and\\|or\\|&&\\|||\\)\\|g?sub!?\\|scan\\|split!?\\)\\s
 *\\(/\\)[^/\n\\\\]*\\(\\\\.[^/\n\\\\]*\\)*\\(/\\)"
+       (4 (7 . ?/))
+       (6 (7 . ?/)))
+      ("^=en\\(d\\)\\_>" 1 "!")
+      ("^\\(=\\)begin\\_>" 1 (ruby-comment-beg-syntax))
+      ;; Currently, the following case is highlighted incorrectly:
+      ;;
+      ;;   <<FOO
+      ;;   FOO
+      ;;   <<BAR
+      ;;   <<BAZ
+      ;;   BAZ
+      ;;   BAR
+      ;;
+      ;; This is because all here-doc beginnings are highlighted before any 
endings,
+      ;; so although <<BAR is properly marked as a beginning, when we get to 
<<BAZ
+      ;; it thinks <<BAR is part of a string so it's marked as well.
+      ;;
+      ;; This may be fixable by modifying ruby-in-here-doc-p to use
+      ;; ruby-in-non-here-doc-string-p rather than syntax-ppss-context,
+      ;; but I don't want to try that until we've got unit tests set up
+      ;; to make sure I don't break anything else.
+      (,(concat ruby-here-doc-beg-re ".*\\(\n\\)")
+       ,(+ 1 (regexp-opt-depth ruby-here-doc-beg-re))
+       (ruby-here-doc-beg-syntax))
+      (,ruby-here-doc-end-re 3 (ruby-here-doc-end-syntax)))
+    "Syntactic keywords for Ruby mode.  See `font-lock-syntactic-keywords'.")
+
+  (defun ruby-comment-beg-syntax ()
+    "Return the syntax cell for a the first character of a =begin.
 See the definition of `ruby-font-lock-syntactic-keywords'.
 
 This returns a comment-delimiter cell as long as the =begin
 isn't in a string or another comment."
-  (when (not (nth 3 (syntax-ppss)))
-    (string-to-syntax "!")))
-
-(unless (functionp 'syntax-ppss)
-  (defun syntax-ppss (&optional pos)
-    (parse-partial-sexp (point-min) (or pos (point)))))
+    (when (not (nth 3 (syntax-ppss)))
+      (string-to-syntax "!")))
+
+  (defun ruby-in-here-doc-p ()
+    "Return whether or not the point is in a heredoc."
+    (save-excursion
+      (let ((old-point (point)) (case-fold-search nil))
+        (beginning-of-line)
+        (catch 'found-beg
+          (while (re-search-backward ruby-here-doc-beg-re nil t)
+            (if (not (or (ruby-in-ppss-context-p 'anything)
+                         (ruby-here-doc-find-end old-point)))
+                (throw 'found-beg t)))))))
+
+  (defun ruby-here-doc-find-end (&optional limit)
+    "Expects the point to be on a line with one or more heredoc openers.
+Returns the buffer position at which all heredocs on the line
+are terminated, or nil if they aren't terminated before the
+buffer position `limit' or the end of the buffer."
+    (save-excursion
+      (beginning-of-line)
+      (catch 'done
+        (let ((eol (save-excursion (end-of-line) (point)))
+              (case-fold-search nil)
+              ;; Fake match data such that (match-end 0) is at eol
+              (end-match-data (progn (looking-at ".*$") (match-data)))
+              beg-match-data end-re)
+          (while (re-search-forward ruby-here-doc-beg-re eol t)
+            (setq beg-match-data (match-data))
+            (setq end-re (ruby-here-doc-end-match))
+
+            (set-match-data end-match-data)
+            (goto-char (match-end 0))
+            (unless (re-search-forward end-re limit t) (throw 'done nil))
+            (setq end-match-data (match-data))
+
+            (set-match-data beg-match-data)
+            (goto-char (match-end 0)))
+          (set-match-data end-match-data)
+          (goto-char (match-end 0))
+          (point)))))
+
+  (defun ruby-here-doc-beg-syntax ()
+    "Return the syntax cell for a line that may begin a heredoc.
+See the definition of `ruby-font-lock-syntactic-keywords'.
+
+This sets the syntax cell for the newline ending the line
+containing the heredoc beginning so that cases where multiple
+heredocs are started on one line are handled correctly."
+    (save-excursion
+      (goto-char (match-beginning 0))
+      (unless (or (ruby-in-ppss-context-p 'non-heredoc)
+                  (ruby-in-here-doc-p))
+        (string-to-syntax "\""))))
+
+  (defun ruby-here-doc-end-syntax ()
+    "Return the syntax cell for a line that may end a heredoc.
+See the definition of `ruby-font-lock-syntactic-keywords'."
+    (let ((pss (syntax-ppss)) (case-fold-search nil))
+      ;; If we aren't in a string, we definitely aren't ending a heredoc,
+      ;; so we can just give up.
+      ;; This means we aren't doing a full-document search
+      ;; every time we enter a character.
+      (when (ruby-in-ppss-context-p 'heredoc pss)
+        (save-excursion
+          (goto-char (nth 8 pss))    ; Go to the beginning of heredoc.
+          (let ((eol (point)))
+            (beginning-of-line)
+            (if (and (re-search-forward (ruby-here-doc-beg-match) eol t) ; If 
there is a heredoc that matches this line...
+                     (not (ruby-in-ppss-context-p 'anything)) ; And that's not 
inside a heredoc/string/comment...
+                     (progn (goto-char (match-end 0)) ; And it's the last 
heredoc on its line...
+                            (not (re-search-forward ruby-here-doc-beg-re eol 
t))))
+                (string-to-syntax "\"")))))))
+
+  (unless (functionp 'syntax-ppss)
+    (defun syntax-ppss (&optional pos)
+      (parse-partial-sexp (point-min) (or pos (point)))))
+  )
 
 (defun ruby-in-ppss-context-p (context &optional ppss)
   (let ((ppss (or ppss (syntax-ppss (point)))))
@@ -1195,10 +1322,7 @@
          ((eq context 'string)
           (nth 3 ppss))
          ((eq context 'heredoc)
-          (and (nth 3 ppss)
-               ;; If it's generic string, it's a heredoc and we don't care
-               ;; See `parse-partial-sexp'
-               (not (numberp (nth 3 ppss)))))
+          (eq ?\n (nth 3 ppss)))
          ((eq context 'non-heredoc)
           (and (ruby-in-ppss-context-p 'anything)
                (not (ruby-in-ppss-context-p 'heredoc))))
@@ -1210,77 +1334,6 @@
                   "context name `" (symbol-name context) "' is unknown"))))
         t)))
 
-(defun ruby-in-here-doc-p ()
-  "Return whether or not the point is in a heredoc."
-  (save-excursion
-    (let ((old-point (point)) (case-fold-search nil))
-      (beginning-of-line)
-      (catch 'found-beg
-        (while (re-search-backward ruby-here-doc-beg-re nil t)
-          (if (not (or (ruby-in-ppss-context-p 'anything)
-                       (ruby-here-doc-find-end old-point)))
-              (throw 'found-beg t)))))))
-
-(defun ruby-here-doc-find-end (&optional limit)
-  "Expects the point to be on a line with one or more heredoc openers.
-Returns the buffer position at which all heredocs on the line
-are terminated, or nil if they aren't terminated before the
-buffer position `limit' or the end of the buffer."
-  (save-excursion
-    (beginning-of-line)
-    (catch 'done
-      (let ((eol (save-excursion (end-of-line) (point)))
-            (case-fold-search nil)
-            ;; Fake match data such that (match-end 0) is at eol
-            (end-match-data (progn (looking-at ".*$") (match-data)))
-            beg-match-data end-re)
-        (while (re-search-forward ruby-here-doc-beg-re eol t)
-          (setq beg-match-data (match-data))
-          (setq end-re (ruby-here-doc-end-match))
-
-          (set-match-data end-match-data)
-          (goto-char (match-end 0))
-          (unless (re-search-forward end-re limit t) (throw 'done nil))
-          (setq end-match-data (match-data))
-
-          (set-match-data beg-match-data)
-          (goto-char (match-end 0)))
-        (set-match-data end-match-data)
-        (goto-char (match-end 0))
-        (point)))))
-
-(defun ruby-here-doc-beg-syntax ()
-  "Return the syntax cell for a line that may begin a heredoc.
-See the definition of `ruby-font-lock-syntactic-keywords'.
-
-This sets the syntax cell for the newline ending the line
-containing the heredoc beginning so that cases where multiple
-heredocs are started on one line are handled correctly."
-  (save-excursion
-    (goto-char (match-beginning 0))
-    (unless (or (ruby-in-ppss-context-p 'non-heredoc)
-                (ruby-in-here-doc-p))
-      (string-to-syntax "|"))))
-
-(defun ruby-here-doc-end-syntax ()
-  "Return the syntax cell for a line that may end a heredoc.
-See the definition of `ruby-font-lock-syntactic-keywords'."
-  (let ((pss (syntax-ppss)) (case-fold-search nil))
-    ;; If we aren't in a string, we definitely aren't ending a heredoc,
-    ;; so we can just give up.
-    ;; This means we aren't doing a full-document search
-    ;; every time we enter a character.
-    (when (ruby-in-ppss-context-p 'heredoc pss)
-      (save-excursion
-        (goto-char (nth 8 pss))  ; Go to the beginning of heredoc.
-        (let ((eol (point)))
-          (beginning-of-line)
-          (if (and (re-search-forward (ruby-here-doc-beg-match) eol t) ; If 
there is a heredoc that matches this line...
-                   (not (ruby-in-ppss-context-p 'anything)) ; And that's not 
inside a heredoc/string/comment...
-                   (progn (goto-char (match-end 0)) ; And it's the last 
heredoc on its line...
-                          (not (re-search-forward ruby-here-doc-beg-re eol 
t))))
-              (string-to-syntax "|")))))))
-
 (if (featurep 'xemacs)
     (put 'ruby-mode 'font-lock-defaults
          '((ruby-font-lock-keywords)
@@ -1377,8 +1430,10 @@
    )
   "Additional expressions to highlight in Ruby mode.")
 
+(defvar electric-indent-chars)
+
 ;;;###autoload
-(defun ruby-mode ()
+(define-derived-mode ruby-mode prog-mode "Ruby"
   "Major mode for editing Ruby scripts.
 \\[ruby-indent-line] properly indents subexpressions of multi-line
 class, module, def, if, while, for, do, and case statements, taking
@@ -1387,27 +1442,22 @@
 The variable `ruby-indent-level' controls the amount of indentation.
 
 \\{ruby-mode-map}"
-  (interactive)
-  (kill-all-local-variables)
-  (use-local-map ruby-mode-map)
-  (setq mode-name "Ruby")
-  (setq major-mode 'ruby-mode)
   (ruby-mode-variables)
 
-  (set (make-local-variable 'indent-line-function)
-       'ruby-indent-line)
   (set (make-local-variable 'imenu-create-index-function)
        'ruby-imenu-create-index)
   (set (make-local-variable 'add-log-current-defun-function)
        'ruby-add-log-current-method)
 
   (add-hook
-   (cond ((boundp 'before-save-hook)
-          (make-local-variable 'before-save-hook)
-          'before-save-hook)
+   (cond ((boundp 'before-save-hook) 'before-save-hook)
          ((boundp 'write-contents-functions) 'write-contents-functions)
          ((boundp 'write-contents-hooks) 'write-contents-hooks))
-   'ruby-mode-set-encoding)
+   'ruby-mode-set-encoding nil 'local)
+
+  (set (make-local-variable 'electric-indent-chars)
+       (append '(?\{ ?\}) (if (boundp 'electric-indent-chars)
+                              (default-value 'electric-indent-chars))))
 
   (set (make-local-variable 'font-lock-defaults)
        '((ruby-font-lock-keywords) nil nil))
@@ -1415,12 +1465,12 @@
        ruby-font-lock-keywords)
   (set (make-local-variable 'font-lock-syntax-table)
        ruby-font-lock-syntax-table)
-  (set (make-local-variable 'font-lock-syntactic-keywords)
-       ruby-font-lock-syntactic-keywords)
 
-  (if (fboundp 'run-mode-hooks)
-      (run-mode-hooks 'ruby-mode-hook)
-    (run-hooks 'ruby-mode-hook)))
+  (if (eval-when-compile (fboundp 'syntax-propertize-rules))
+      (set (make-local-variable 'syntax-propertize-function)
+           #'ruby-syntax-propertize-function)
+    (set (make-local-variable 'font-lock-syntactic-keywords)
+         ruby-font-lock-syntactic-keywords)))
 
 ;;; Invoke ruby-mode when appropriate
 

=== modified file 'lisp/progmodes/sh-script.el'
--- a/lisp/progmodes/sh-script.el       2010-08-26 15:26:33 +0000
+++ b/lisp/progmodes/sh-script.el       2010-09-10 23:13:42 +0000
@@ -939,7 +939,6 @@
 ;; These are used for the syntax table stuff (derived from cperl-mode).
 ;; Note: parse-sexp-lookup-properties must be set to t for it to work.
 (defconst sh-st-punc (string-to-syntax "."))
-(defconst sh-st-symbol (string-to-syntax "_"))
 (defconst sh-here-doc-syntax (string-to-syntax "|")) ;; generic string
 
 (defconst sh-escaped-line-re
@@ -957,7 +956,7 @@
 (defvar sh-here-doc-re sh-here-doc-open-re)
 (make-variable-buffer-local 'sh-here-doc-re)
 
-(defun sh-font-lock-close-heredoc (bol eof indented)
+(defun sh-font-lock-close-heredoc (bol eof indented eol)
   "Determine the syntax of the \\n after an EOF.
 If non-nil INDENTED indicates that the EOF was indented."
   (let* ((eof-re (if eof (regexp-quote eof) ""))
@@ -971,6 +970,8 @@
         (ere (concat "^" (if indented "[ \t]*") eof-re "\n"))
         (start (save-excursion
                  (goto-char bol)
+                  ;; FIXME: will incorrectly find a <<EOF embedded inside
+                  ;; the heredoc.
                  (re-search-backward (concat sre "\\|" ere) nil t))))
     ;; If subgroup 1 matched, we found an open-heredoc, otherwise we first
     ;; found a close-heredoc which makes the current close-heredoc inoperant.
@@ -990,7 +991,7 @@
                     (sh-in-comment-or-string (point)))))
          ;; No <<EOF2 found after our <<.
          (= (point) start)))
-      sh-here-doc-syntax)
+      (put-text-property eol (1+ eol) 'syntax-table sh-here-doc-syntax))
      ((not (or start (save-excursion (re-search-forward sre nil t))))
       ;; There's no <<EOF either before or after us,
       ;; so we should remove ourselves from font-lock's keywords.
@@ -1000,7 +1001,7 @@
                    (regexp-opt sh-here-doc-markers t) "\\(\n\\)"))
       nil))))
 
-(defun sh-font-lock-open-heredoc (start string)
+(defun sh-font-lock-open-heredoc (start string eol)
   "Determine the syntax of the \\n after a <<EOF.
 START is the position of <<.
 STRING is the actual word used as delimiter (e.g. \"EOF\").
@@ -1030,13 +1031,8 @@
           ;; Don't bother fixing it now, but place a multiline property so
           ;; that when jit-lock-context-* refontifies the rest of the
           ;; buffer, it also refontifies the current line with it.
-          (put-text-property start (point) 'font-lock-multiline t)))
-    sh-here-doc-syntax))
-
-(defun sh-font-lock-here-doc (limit)
-  "Search for a heredoc marker."
-  ;; This looks silly, but it's because `sh-here-doc-re' keeps changing.
-  (re-search-forward sh-here-doc-re limit t))
+          (put-text-property start (point) 'syntax-multiline t)))
+    (put-text-property eol (1+ eol) 'syntax-table sh-here-doc-syntax)))
 
 (defun sh-font-lock-quoted-subshell (limit)
   "Search for a subshell embedded in a string.
@@ -1045,9 +1041,7 @@
   ;; FIXME: This can (and often does) match multiple lines, yet it makes no
   ;; effort to handle multiline cases correctly, so it ends up being
   ;; rather flakey.
-  (when (and (re-search-forward 
"\"\\(?:\\(?:.\\|\n\\)*?[^\\]\\(?:\\\\\\\\\\)*\\)??\\(\\$(\\|`\\)" limit t)
-             ;; Make sure the " we matched is an opening quote.
-            (eq ?\" (nth 3 (syntax-ppss))))
+  (when (eq ?\" (nth 3 (syntax-ppss))) ; Check we matched an opening quote.
     ;; bingo we have a $( or a ` inside a ""
     (let ((char (char-after (point)))
           ;; `state' can be: double-quote, backquote, code.
@@ -1082,8 +1076,7 @@
                  (double-quote nil)
                  (t (setq state (pop states)))))
           (t (error "Internal error in sh-font-lock-quoted-subshell")))
-        (forward-char 1)))
-    t))
+        (forward-char 1)))))
 
 
 (defun sh-is-quoted-p (pos)
@@ -1122,7 +1115,7 @@
     (when (progn (backward-char 2)
                  (if (> start (line-end-position))
                      (put-text-property (point) (1+ start)
-                                        'font-lock-multiline t))
+                                        'syntax-multiline t))
                  ;; FIXME: The `in' may just be a random argument to
                  ;; a normal command rather than the real `in' keyword.
                  ;; I.e. we should look back to try and find the
@@ -1136,40 +1129,44 @@
       sh-st-punc
     nil))
 
-(defun sh-font-lock-flush-syntax-ppss-cache (limit)
-  ;; This should probably be a standard function provided by font-lock.el
-  ;; (or syntax.el).
-  (syntax-ppss-flush-cache (point))
-  (goto-char limit)
-  nil)
-
-(defconst sh-font-lock-syntactic-keywords
-  ;; A `#' begins a comment when it is unquoted and at the beginning of a
-  ;; word.  In the shell, words are separated by metacharacters.
-  ;; The list of special chars is taken from the single-unix spec
-  ;; of the shell command language (under `quoting') but with `$' removed.
-  `(("[^|&;<>()`\\\"' \t\n]\\(#+\\)" 1 ,sh-st-symbol)
-    ;; In a '...' the backslash is not escaping.
-    ("\\(\\\\\\)'" (1 (sh-font-lock-backslash-quote)))
-    ;; The previous rule uses syntax-ppss, but the subsequent rules may
-    ;; change the syntax, so we have to tell syntax-ppss that the states it
-    ;; has just computed will need to be recomputed.
-    (sh-font-lock-flush-syntax-ppss-cache)
-    ;; Make sure $@ and $? are correctly recognized as sexps.
-    ("\\$\\(address@hidden)" 1 ,sh-st-symbol)
-    ;; Find HEREDOC starters and add a corresponding rule for the ender.
-    (sh-font-lock-here-doc
-     (2 (sh-font-lock-open-heredoc
-        (match-beginning 0) (match-string 1)) nil t)
-     (5 (sh-font-lock-close-heredoc
-        (match-beginning 0) (match-string 4)
-         (and (match-beginning 3) (/= (match-beginning 3) (match-end 3))))
-      nil t))
-    ;; Distinguish the special close-paren in `case'.
-    (")" 0 (sh-font-lock-paren (match-beginning 0)))
-    ;; highlight (possibly nested) subshells inside "" quoted regions 
correctly.
-    ;; This should be at the very end because it uses syntax-ppss.
-    (sh-font-lock-quoted-subshell)))
+(defun sh-syntax-propertize-function (start end)
+  (goto-char start)
+  (while (prog1
+             (re-search-forward sh-here-doc-re end 'move)
+           (save-excursion
+             (save-match-data
+               (funcall
+                (syntax-propertize-rules
+                 ;; A `#' begins a comment when it is unquoted and at the
+                 ;; beginning of a word.  In the shell, words are separated by
+                 ;; metacharacters.  The list of special chars is taken from
+                 ;; the single-unix spec of the shell command language (under
+                 ;; `quoting') but with `$' removed.
+                 ("[^|&;<>()`\\\"' \t\n]\\(#+\\)" (1 "_"))
+                 ;; In a '...' the backslash is not escaping.
+                 ("\\(\\\\\\)'" (1 (sh-font-lock-backslash-quote)))
+                 ;; Make sure $@ and $? are correctly recognized as sexps.
+                 ("\\$\\(address@hidden)" (1 "_"))
+                 ;; Distinguish the special close-paren in `case'.
+                 (")" (0 (sh-font-lock-paren (match-beginning 0))))
+                 ;; Highlight (possibly nested) subshells inside "" quoted
+                 ;; regions correctly.
+                 
("\"\\(?:\\(?:.\\|\n\\)*?[^\\]\\(?:\\\\\\\\\\)*\\)??\\(\\$(\\|`\\)"
+                  (1 (ignore
+                      ;; Save excursion because we want to also apply other
+                      ;; syntax-propertize rules within the affected region.
+                      (save-excursion
+                        (sh-font-lock-quoted-subshell end))))))
+                (prog1 start (setq start (point))) (point)))))
+    (if (match-beginning 2)
+        ;; FIXME: actually, once we see an heredoc opener, we should just
+        ;; search for its ender without propertizing anything in it.
+        (sh-font-lock-open-heredoc
+        (match-beginning 0) (match-string 1) (match-beginning 2))
+      (sh-font-lock-close-heredoc
+       (match-beginning 0) (match-string 4)
+       (and (match-beginning 3) (/= (match-beginning 3) (match-end 3)))
+       (match-beginning 5)))))
 
 (defun sh-font-lock-syntactic-face-function (state)
   (let ((q (nth 3 state)))
@@ -1553,9 +1550,12 @@
           sh-font-lock-keywords-1 sh-font-lock-keywords-2)
          nil nil
          ((?/ . "w") (?~ . "w") (?. . "w") (?- . "w") (?_ . "w")) nil
-         (font-lock-syntactic-keywords . sh-font-lock-syntactic-keywords)
          (font-lock-syntactic-face-function
           . sh-font-lock-syntactic-face-function)))
+  (set (make-local-variable 'syntax-propertize-function)
+       #'sh-syntax-propertize-function)
+  (add-hook 'syntax-propertize-extend-region-functions
+            #'syntax-propertize-multiline 'append 'local)
   (set (make-local-variable 'skeleton-pair-alist) '((?` _ ?`)))
   (set (make-local-variable 'skeleton-pair-filter-function) 'sh-quoted-p)
   (set (make-local-variable 'skeleton-further-elements)

=== modified file 'lisp/progmodes/simula.el'
--- a/lisp/progmodes/simula.el  2010-07-27 16:37:25 +0000
+++ b/lisp/progmodes/simula.el  2010-09-10 23:13:42 +0000
@@ -163,17 +163,18 @@
 (defvar simula-mode-syntax-table nil
   "Syntax table in SIMULA mode buffers.")
 
-(defconst simula-font-lock-syntactic-keywords
-  `(;; `comment' directive.
-    ("\\<\\(c\\)omment\\>" 1 "<")
-    ;; end comments
-    (,(concat "\\<end\\>\\([^;\n]\\).*?\\(\n\\|\\(.\\)\\(;\\|"
-             (regexp-opt '("end" "else" "when" "otherwise"))
-             "\\)\\)")
-     (1 "< b")
-     (3 "> b" nil t))
-    ;; non-quoted single-quote char.
-    ("'\\('\\)'" 1 ".")))
+(defconst simula-syntax-propertize-function
+  (syntax-propertize-rules
+   ;; `comment' directive.
+   ("\\<\\(c\\)omment\\>" (1 "<"))
+   ;; end comments
+   ((concat "\\<end\\>\\([^;\n]\\).*?\\(\n\\|\\(.\\)\\(;\\|"
+            (regexp-opt '("end" "else" "when" "otherwise"))
+            "\\)\\)")
+    (1 "< b")
+    (3 "> b"))
+   ;; non-quoted single-quote char.
+   ("'\\('\\)'" (1 "."))))
 
 ;; Regexps written with help from Alf-Ivar Holm <address@hidden>.
 (defconst simula-font-lock-keywords-1
@@ -396,8 +397,9 @@
   (setq font-lock-defaults
        '((simula-font-lock-keywords simula-font-lock-keywords-1
           simula-font-lock-keywords-2 simula-font-lock-keywords-3)
-         nil t ((?_ . "w")) nil
-         (font-lock-syntactic-keywords . simula-font-lock-syntactic-keywords)))
+         nil t ((?_ . "w"))))
+  (set (make-local-variable 'syntax-propertize-function)
+       simula-syntax-propertize-function)
   (abbrev-mode 1))
 
 (defun simula-indent-exp ()

=== modified file 'lisp/progmodes/tcl.el'
--- a/lisp/progmodes/tcl.el     2010-07-27 16:37:25 +0000
+++ b/lisp/progmodes/tcl.el     2010-09-10 23:13:42 +0000
@@ -411,9 +411,10 @@
 `tcl-typeword-list', and `tcl-keyword-list' by the function
 `tcl-set-font-lock-keywords'.")
 
-(defvar tcl-font-lock-syntactic-keywords
-  ;; Mark the few `#' that are not comment-markers.
-  '(("[^;[{ \t\n][ \t]*\\(#\\)" (1 ".")))
+(defconst tcl-syntax-propertize-function
+  (syntax-propertize-rules
+   ;; Mark the few `#' that are not comment-markers.
+   ("[^;[{ \t\n][ \t]*\\(#\\)" (1 ".")))
   "Syntactic keywords for `tcl-mode'.")
 
 ;; FIXME need some way to recognize variables because array refs look
@@ -593,9 +594,9 @@
   (set (make-local-variable 'outline-level) 'tcl-outline-level)
 
   (set (make-local-variable 'font-lock-defaults)
-       '(tcl-font-lock-keywords nil nil nil beginning-of-defun
-        (font-lock-syntactic-keywords . tcl-font-lock-syntactic-keywords)
-        (parse-sexp-lookup-properties . t)))
+       '(tcl-font-lock-keywords nil nil nil beginning-of-defun))
+  (set (make-local-variable 'syntax-propertize-function)
+       tcl-syntax-propertize-function)
 
   (set (make-local-variable 'imenu-generic-expression)
        tcl-imenu-generic-expression)

=== modified file 'lisp/progmodes/vhdl-mode.el'
--- a/lisp/progmodes/vhdl-mode.el       2010-05-25 02:11:08 +0000
+++ b/lisp/progmodes/vhdl-mode.el       2010-09-10 23:13:42 +0000
@@ -4693,8 +4693,15 @@
   (set (make-local-variable 'font-lock-defaults)
        (list
        '(nil vhdl-font-lock-keywords) nil
-       (not vhdl-highlight-case-sensitive) '((?\_ . "w")) 'beginning-of-line
-       '(font-lock-syntactic-keywords . vhdl-font-lock-syntactic-keywords)))
+       (not vhdl-highlight-case-sensitive) '((?\_ . "w")) 'beginning-of-line))
+  (if (eval-when-compile (fboundp 'syntax-propertize-rules))
+      (set (make-local-variable 'syntax-propertize-function)
+           (syntax-propertize-rules
+            ;; Mark single quotes as having string quote syntax in
+            ;; 'c' instances.
+            ("\\(\'\\).\\(\'\\)" (1 "\"'") (2 "\"'"))))
+    (set (make-local-variable 'font-lock-syntactic-keywords)
+         vhdl-font-lock-syntactic-keywords))
   (unless vhdl-emacs-21
     (set (make-local-variable 'font-lock-support-mode) 'lazy-lock-mode)
     (set (make-local-variable 'lazy-lock-defer-contextually) nil)
@@ -12914,10 +12921,9 @@
   "Re-initialize fontification and fontify buffer."
   (interactive)
   (setq font-lock-defaults
-       (list
-        'vhdl-font-lock-keywords nil
-        (not vhdl-highlight-case-sensitive) '((?\_ . "w")) 'beginning-of-line
-        '(font-lock-syntactic-keywords . vhdl-font-lock-syntactic-keywords)))
+       `(vhdl-font-lock-keywords
+          nil ,(not vhdl-highlight-case-sensitive) ((?\_ . "w"))
+          beginning-of-line))
   (when (fboundp 'font-lock-unset-defaults)
     (font-lock-unset-defaults))                ; not implemented in XEmacs
   (font-lock-set-defaults)

=== modified file 'lisp/textmodes/bibtex.el'
--- a/lisp/textmodes/bibtex.el  2010-09-06 16:38:03 +0000
+++ b/lisp/textmodes/bibtex.el  2010-09-10 23:13:42 +0000
@@ -3027,12 +3027,14 @@
                 ;; brace-delimited ones
                 )
          nil
-         (font-lock-syntactic-keywords . bibtex-font-lock-syntactic-keywords)
          (font-lock-extra-managed-props . (category))
         (font-lock-mark-block-function
          . (lambda ()
               (set-mark (bibtex-end-of-entry))
              (bibtex-beginning-of-entry)))))
+  (set (make-local-variable 'syntax-propertize-function)
+       (syntax-propertize-via-font-lock
+        bibtex-font-lock-syntactic-keywords))
   (setq imenu-generic-expression
         (list (list nil bibtex-entry-head bibtex-key-in-head))
         imenu-case-fold-search t)

=== modified file 'lisp/textmodes/reftex.el'
--- a/lisp/textmodes/reftex.el  2010-05-03 02:29:46 +0000
+++ b/lisp/textmodes/reftex.el  2010-09-10 23:13:42 +0000
@@ -599,7 +599,6 @@
 (defvar font-lock-mode)
 (defvar font-lock-keywords)
 (defvar font-lock-fontify-region-function)
-(defvar font-lock-syntactic-keywords)
 
 ;;; =========================================================================
 ;;;

=== modified file 'lisp/textmodes/sgml-mode.el'
--- a/lisp/textmodes/sgml-mode.el       2010-05-25 02:11:08 +0000
+++ b/lisp/textmodes/sgml-mode.el       2010-09-10 23:13:42 +0000
@@ -293,11 +293,12 @@
 (defvar sgml-font-lock-keywords sgml-font-lock-keywords-1
   "*Rules for highlighting SGML code.  See also `sgml-tag-face-alist'.")
 
-(defvar sgml-font-lock-syntactic-keywords
+(defconst sgml-syntax-propertize-function
+  (syntax-propertize-rules
   ;; Use the `b' style of comments to avoid interference with the -- ... --
   ;; comments recognized when `sgml-specials' includes ?-.
   ;; FIXME: beware of <!--> blabla <!--> !!
-  '(("\\(<\\)!--" (1 "< b"))
+   ("\\(<\\)!--" (1 "< b"))
     ("--[ \t\n]*\\(>\\)" (1 "> b"))
     ;; Double quotes outside of tags should not introduce strings.
     ;; Be careful to call `syntax-ppss' on a position before the one we're
@@ -477,9 +478,9 @@
        '((sgml-font-lock-keywords
           sgml-font-lock-keywords-1
           sgml-font-lock-keywords-2)
-         nil t nil nil
-         (font-lock-syntactic-keywords
-          . sgml-font-lock-syntactic-keywords)))
+         nil t))
+  (set (make-local-variable 'syntax-propertize-function)
+       sgml-syntax-propertize-function)
   (set (make-local-variable 'facemenu-add-face-function)
        'sgml-mode-facemenu-add-face-function)
   (set (make-local-variable 'sgml-xml-mode) (sgml-xml-guess))

=== modified file 'lisp/textmodes/tex-mode.el'
--- a/lisp/textmodes/tex-mode.el        2010-05-13 21:52:20 +0000
+++ b/lisp/textmodes/tex-mode.el        2010-09-10 23:13:42 +0000
@@ -488,7 +488,7 @@
           ;; (arg 
"\\(?:{\\(\\(?:[^{}\\]+\\|\\\\.\\|{[^}]*}\\)+\\)\\|\\\\[a-z*]+\\)"))
           (arg "{\\(\\(?:[^{}\\]+\\|\\\\.\\|{[^}]*}\\)+\\)"))
       (list
-       ;; font-lock-syntactic-keywords causes the \ of \end{verbatim} to be
+       ;; tex-font-lock-syntactic-keywords causes the \ of \end{verbatim} to be
        ;; highlighted as tex-verbatim face.  Let's undo that.
        ;; This is ugly and brittle :-(  --Stef
        '("^\\(\\\\\\)end" (1 (get-text-property (match-end 1) 'face) t))
@@ -655,6 +655,7 @@
     ;;     line is re-font-locked on its own.
     ;; There's a hack in tex-font-lock-keywords-1 to remove the verbatim
     ;; face from the \ but C-M-f still jumps to the wrong spot :-(  --Stef
+    ;; FIXME: See gud.el for an example of a solution to a similar problem.
     (eval . `(,(concat "^\\(\\\\\\)end *{"
                        (regexp-opt tex-verbatim-environments t)
                        "}\\(.?\\)") (1 "|") (3 "<")))
@@ -1163,10 +1164,9 @@
         (font-lock-syntactic-face-function
          . tex-font-lock-syntactic-face-function)
         (font-lock-unfontify-region-function
-         . tex-font-lock-unfontify-region)
-        (font-lock-syntactic-keywords
-         . tex-font-lock-syntactic-keywords)
-        (parse-sexp-lookup-properties . t)))
+         . tex-font-lock-unfontify-region)))
+  (set (make-local-variable 'syntax-propertize-function)
+       (syntax-propertize-via-font-lock tex-font-lock-syntactic-keywords))
   ;; TABs in verbatim environments don't do what you think.
   (set (make-local-variable 'indent-tabs-mode) nil)
   ;; Other vars that should be buffer-local.
@@ -2850,12 +2850,12 @@
              (mapcar
               (lambda (x)
                 (case (car-safe x)
-                  (font-lock-syntactic-keywords
-                   (cons (car x) 'doctex-font-lock-syntactic-keywords))
                   (font-lock-syntactic-face-function
                    (cons (car x) 'doctex-font-lock-syntactic-face-function))
                   (t x)))
-              (cdr font-lock-defaults)))))
+              (cdr font-lock-defaults))))
+  (set (make-local-variable 'syntax-propertize-function)
+       (syntax-propertize-via-font-lock doctex-font-lock-syntactic-keywords)))
 
 (run-hooks 'tex-mode-load-hook)
 

=== modified file 'lisp/textmodes/texinfo.el'
--- a/lisp/textmodes/texinfo.el 2010-01-13 08:35:10 +0000
+++ b/lisp/textmodes/texinfo.el 2010-09-10 23:13:42 +0000
@@ -310,10 +310,11 @@
     ("Chapters" "address@hidden \t]+\\(.*\\)$" 1))
   "Imenu generic expression for Texinfo mode.  See 
`imenu-generic-expression'.")
 
-(defvar texinfo-font-lock-syntactic-keywords
-  '(("\\(@\\)c\\(omment\\)?\\>" (1 "<"))
-    ("^\\(@\\)ignore\\>" (1 "< b"))
-    ("address@hidden ignore\\(\n\\)" (1 "> b")))
+(defconst texinfo-syntax-propertize-function
+  (syntax-propertize-rules
+   ("\\(@\\)c\\(omment\\)?\\>" (1 "<"))
+   ("^\\(@\\)ignore\\>" (1 "< b"))
+   ("address@hidden ignore\\(\n\\)" (1 "> b")))
   "Syntactic keywords to catch comment delimiters in `texinfo-mode'.")
 
 (defconst texinfo-environments
@@ -600,9 +601,9 @@
   (setq imenu-case-fold-search nil)
   (make-local-variable 'font-lock-defaults)
   (setq font-lock-defaults
-       '(texinfo-font-lock-keywords nil nil nil backward-paragraph
-                                    (font-lock-syntactic-keywords
-                                     . texinfo-font-lock-syntactic-keywords)))
+       '(texinfo-font-lock-keywords nil nil nil backward-paragraph))
+  (set (make-local-variable 'syntax-propertize-function)
+       texinfo-syntax-propertize-function)
   (set (make-local-variable 'parse-sexp-lookup-properties) t)
 
   ;; Outline settings.

=== modified file 'test/ChangeLog'
--- a/test/ChangeLog    2010-08-30 20:34:52 +0000
+++ b/test/ChangeLog    2010-09-10 23:13:42 +0000
@@ -1,3 +1,7 @@
+2010-09-10  Stefan Monnier  <address@hidden>
+
+       * indent/octave.m: Remove some `fixindent' not needed any more.
+
 2010-08-30  Stefan Monnier  <address@hidden>
 
        * indent/octave.m: New file.

=== modified file 'test/indent/octave.m'
--- a/test/indent/octave.m      2010-08-30 20:34:52 +0000
+++ b/test/indent/octave.m      2010-09-10 23:13:42 +0000
@@ -1415,7 +1415,7 @@
     endfor                                   # fixindent
 
     ## Search all C++ source files for PKG commands.
-    lst = dir (fullfile (packdir, "src", "*.cc")); # fixindent
+    lst = dir (fullfile (packdir, "src", "*.cc"));
     for i = 1:length (lst)
       nam = fullfile (packdir, "src", lst(i).name);
       fwrite (archfid, extract_pkg (nam, ['^//* *' nm ': *(.*)$']));
@@ -1451,10 +1451,10 @@
         unlink (archpkg);
       endif
     endif
-  endif                         # fixindent
-endfunction                     # fixindent
+  endif
+endfunction
 
-function copy_files (desc, packdir, global_install) # fixindent
+function copy_files (desc, packdir, global_install)
   ## Create the installation directory.
   if (! exist (desc.dir, "dir"))
     [status, output] = mkdir (desc.dir);


reply via email to

[Prev in Thread] Current Thread [Next in Thread]