[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: SMIE examples or guides

From: Björn Lindqvist
Subject: Re: SMIE examples or guides
Date: Fri, 1 Jul 2016 21:28:01 +0200

I managed to write the forward and backward tokenization functions:

(defun factor-smie-token (dir)
  (pcase dir
    ('forward (forward-comment (point-max)))
    ('backward (forward-comment (- (point)))))
  (let ((tok (buffer-substring-no-properties
              (let ((syntax "w_\\\""))
                (pcase dir
                  ('forward (skip-syntax-forward syntax))
                  ('backward (skip-syntax-backward syntax)))
    ;; Normalizes different indent starters.
    (cond ((string-match factor-smie-indents-regex tok) ":")
          (t tok))))

(defun factor-smie-forward-token ()
  (factor-smie-token 'forward))

(defun factor-smie-backward-token ()
  (factor-smie-token 'backward))

It works in 99% of the cases. But since almost any character can be
part of a token, it doesn't work perfectly. E.g te]st(ab3 would be a
perfectly valid variable name. I tried changing the syntax to
"w_()\\\"" and that fixes the tokenization but then I lose the useful
automatic indentation smie adds to opening- and closing bracket
characters. E.g I'm happy that smie indents:


It's very nice. But the following two lines are not right:


They should be:


2016-07-01 9:13 GMT+02:00 Stefan Monnier <address@hidden>:
>> Many thanks! That appears to work. But there is one problem. In a
>> buffer with just this content:
>>     [
>>     \
>>     ]
>> I get the error: (error "Bumped into unknown token")
> You should either give a different syntax to the \ char in the
> syntax-table, or change the :forward-token and :backward-token functions
> so they do something with \
>>       (smie-bnf->prec2
>>        '((exp (("HELLO" "HALLO" "CIAO" "SALUT") exp "BYE"))))))
> One way to do that is:
>        (smie-bnf->prec2
>         '((exp ("HELLO" exp "BYE")
>                ("HALLO" exp "BYE")
>                ("CIAO" exp "BYE")
>                ("SALUT" exp "BYE"))))))
> Another is to change the :forward-token and :backward-token functions so
> they return the same string (e.g. "HELLO") when tokenizing any one of those.
>         Stefan

mvh/best regards Björn Lindqvist

reply via email to

[Prev in Thread] Current Thread [Next in Thread]