[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[elpa] externals/parser-generator a4c5d9f 429/434: Syntax-directed trans
|
From: |
ELPA Syncer |
|
Subject: |
[elpa] externals/parser-generator a4c5d9f 429/434: Syntax-directed translations now have access to terminal data if any is available |
|
Date: |
Mon, 29 Nov 2021 16:00:30 -0500 (EST) |
branch: externals/parser-generator
commit a4c5d9f1a1f5d9cf845244def341d739cfaec116
Author: Christian Johansson <christian@cvj.se>
Commit: Christian Johansson <christian@cvj.se>
Syntax-directed translations now have access to terminal data if any is
available
---
docs/Syntax-Analysis.md | 6 +--
parser-generator-lr-export.el | 31 ++++++++----
parser-generator-lr.el | 36 ++++++++++----
test/parser-generator-lr-export-test.el | 57 +++++++++++++++++++---
test/parser-generator-lr-test.el | 85 ++++++++++++++++++++++++++-------
5 files changed, 172 insertions(+), 43 deletions(-)
diff --git a/docs/Syntax-Analysis.md b/docs/Syntax-Analysis.md
index a4c0816..836cde6 100644
--- a/docs/Syntax-Analysis.md
+++ b/docs/Syntax-Analysis.md
@@ -117,20 +117,20 @@ A optional translation is defined as a lambda function as
the last element of a
(require 'parser-generator)
(parser-generator-set-grammar
- '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b" (lambda(args) (nreverse args))))
(S e)) Sp))
+ '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b" (lambda(args _terminals)
(nreverse args)))) (S e)) Sp))
```
You cannot have a SDT + SA on the same production right-hand side, just one or
the other.
### Semantic actions (SA)
-A optional semantic action is defined as a lambda function as the last element
of a production right-hand-side, example:
+A optional semantic action is defined as a lambda function as the last element
of a production right-hand-side, two arguments were first is the value of the
symbols in the rule and the second is the terminal values for the same symbols,
example:
```emacs-lisp
(require 'parser-generator)
(parser-generator-set-grammar
- '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b" (lambda(args) (nreverse args))))
(S e)) Sp))
+ '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b" (lambda(args _terminals)
(nreverse args)))) (S e)) Sp))
```
You cannot have a SDT + SA on the same production right-hand side, just one or
the other.
diff --git a/parser-generator-lr-export.el b/parser-generator-lr-export.el
index 8b34a87..2cc54ca 100644
--- a/parser-generator-lr-export.el
+++ b/parser-generator-lr-export.el
@@ -731,7 +731,8 @@
(push production-number output)
(when perform-sdt
- (let ((popped-items-meta-contents))
+ (let ((popped-items-meta-contents)
+ (popped-items-terminals))
(setq
popped-items-contents
(reverse popped-items-contents))
@@ -741,10 +742,14 @@
(listp popped-item)
(cdr popped-item))
;; If item is a terminal, use it's literal
value
- (push
- (%s-lex-analyzer--get-function
- popped-item)
- popped-items-meta-contents)"
+ (progn
+ (push
+ (%s-lex-analyzer--get-function
+ popped-item)
+ popped-items-meta-contents)
+ (push
+ popped-item
+ popped-items-terminals))"
namespace
namespace
namespace
@@ -776,19 +781,28 @@
(push
symbol-translation
popped-items-meta-contents)
+ (push
+ nil
+ popped-items-terminals)
(puthash
temp-hash-key
symbol-translations
translation-symbol-table)))
(push
nil
- popped-items-meta-contents)))))
+ popped-items-meta-contents)
+ (push
+ nil
+ popped-items-terminals)))))
;; If we just have one argument, pass it as a
instead of a list
(when (= (length popped-items-meta-contents) 1)
(setq
popped-items-meta-contents
- (car popped-items-meta-contents)))
+ (car popped-items-meta-contents))
+ (setq
+ popped-items-terminals
+ (car popped-items-terminals)))
;; Perform translation at reduction if specified
(if
@@ -798,7 +812,8 @@
(funcall
(%s--get-grammar-translation-by-number
production-number)
- popped-items-meta-contents)))"
+ popped-items-meta-contents
+ popped-items-terminals)))"
namespace
namespace))
diff --git a/parser-generator-lr.el b/parser-generator-lr.el
index a79a046..5ac531d 100644
--- a/parser-generator-lr.el
+++ b/parser-generator-lr.el
@@ -1972,7 +1972,8 @@
(push production-number output)
(when perform-sdt
- (let ((popped-items-meta-contents))
+ (let ((popped-items-meta-contents)
+ (popped-items-terminals))
(setq
popped-items-contents
(reverse popped-items-contents))
@@ -1987,10 +1988,14 @@
(cdr popped-item))
;; If item is a terminal, use it's literal
value
- (push
-
(parser-generator-lex-analyzer--get-function
- popped-item)
- popped-items-meta-contents)
+ (progn
+ (push
+
(parser-generator-lex-analyzer--get-function
+ popped-item)
+ popped-items-meta-contents)
+ (push
+ popped-item
+ popped-items-terminals))
;; If item is a non-terminal
(let ((temp-hash-key
@@ -2012,26 +2017,36 @@
(push
symbol-translation
popped-items-meta-contents)
+ (push
+ nil
+ popped-items-terminals)
(puthash
temp-hash-key
symbol-translations
translation-symbol-table)))
(push
nil
- popped-items-meta-contents)))))
+ popped-items-meta-contents)
+ (push
+ nil
+ popped-items-terminals)))))
;; If we just have one argument, pass it as a
instead of a list
(when (= (length popped-items-meta-contents) 1)
(setq
popped-items-meta-contents
- (car popped-items-meta-contents)))
+ (car popped-items-meta-contents))
+ (setq
+ popped-items-terminals
+ (car popped-items-terminals)))
(parser-generator--debug
(message
- "Production arguments: %s -> %s = %s"
+ "Production arguments: %s -> %s = %s (%S)"
production-lhs
production-rhs
- popped-items-meta-contents))
+ popped-items-meta-contents
+ popped-items-terminals))
;; Perform translation at reduction if specified
(if
@@ -2041,7 +2056,8 @@
(funcall
(parser-generator--get-grammar-translation-by-number
production-number)
- popped-items-meta-contents)))
+ popped-items-meta-contents
+ popped-items-terminals)))
(parser-generator--debug
(message
"translation-symbol-table: %S = %S
(processed)"
diff --git a/test/parser-generator-lr-export-test.el
b/test/parser-generator-lr-export-test.el
index ceefa20..75999b8 100644
--- a/test/parser-generator-lr-export-test.el
+++ b/test/parser-generator-lr-export-test.el
@@ -24,9 +24,12 @@
("a" "b")
(
(Sp S)
- (S (S "a" S "b" (lambda(args) (let ((list "")) (dolist (item args)
(when item (setq list (format "%s%s" item list)))) list))))
+ (S (S "a" S "b" (lambda(args _terminals) (let ((list "")) (dolist
(item args) (when item (setq list (format "%s%s" item list)))) list))))
(S e)
- ) Sp))
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -146,7 +149,17 @@
;; Generate parser
(parser-generator-set-grammar
- '((Sp S) (a b) ((Sp S) (S (S a S b)) (S e)) Sp))
+ '(
+ (Sp S)
+ (a b)
+ (
+ (Sp S)
+ (S (S a S b))
+ (S e)
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -199,7 +212,18 @@
(switch-to-buffer buffer)
(insert "aabb")
- (parser-generator-set-grammar '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b"
(lambda(args) (let ((list "")) (dolist (item args) (when item (setq list
(format "%s%s" item list)))) list)))) (S e)) Sp))
+ (parser-generator-set-grammar
+ '(
+ (Sp S)
+ ("a" "b")
+ (
+ (Sp S)
+ (S (S "a" S "b" (lambda(args _terminals) (let ((list "")) (dolist
(item args) (when item (setq list (format "%s%s" item list)))) list))))
+ (S e)
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -259,7 +283,17 @@
(insert "1+1")
(parser-generator-set-grammar
- '((S E B) ("*" "+" "0" "1") ((S (E $)) (E (E "*" B) (E "+" B) (B)) (B ("0")
("1"))) S))
+ '(
+ (S E B)
+ ("*" "+" "0" "1")
+ (
+ (S (E $))
+ (E (E "*" B) (E "+" B) (B))
+ (B ("0") ("1"))
+ )
+ S
+ )
+ )
(parser-generator-set-look-ahead-number 0)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -319,7 +353,18 @@
(switch-to-buffer buffer)
(insert "aabb")
- (parser-generator-set-grammar '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b"
(lambda(args) (let ((list "")) (dolist (item args) (when item (setq list
(format "%s%s" item list)))) list)))) (S e)) Sp))
+ (parser-generator-set-grammar
+ '(
+ (Sp S)
+ ("a" "b")
+ (
+ (Sp S)
+ (S (S "a" S "b" (lambda(args _terminals) (let ((list "")) (dolist
(item args) (when item (setq list (format "%s%s" item list)))) list))))
+ (S e)
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
diff --git a/test/parser-generator-lr-test.el b/test/parser-generator-lr-test.el
index 6956cb7..07f7f4e 100644
--- a/test/parser-generator-lr-test.el
+++ b/test/parser-generator-lr-test.el
@@ -393,19 +393,19 @@
(start input)
(input
%empty
- (input line (lambda(args) (nth 1 args))))
+ (input line (lambda(args _terminals) (nth 1 args))))
(line
"\n"
- (exp "\n" (lambda(args) (nth 0 args))))
+ (exp "\n" (lambda(args _terminals) (nth 0 args))))
(exp
NUM
- (exp "+" exp (lambda(args) (+ (float (nth 0 args)) (nth 2 args))))
- (exp "-" exp (lambda(args) (- (float (nth 0 args)) (nth 2 args))))
- (exp "*" exp (lambda(args) (* (float (nth 0 args)) (nth 2 args))))
- (exp "/" exp (lambda(args) (/ (float (nth 0 args)) (nth 2 args))))
- ("-" exp %prec NEG (lambda(args) (- (float (nth 1 args)))))
- (exp "^" exp (lambda(args) (expt (float (nth 0 args)) (nth 2 args))))
- ("(" exp ")" (lambda(args) (nth 1 args)))))
+ (exp "+" exp (lambda(args _terminals) (+ (float (nth 0 args)) (nth 2
args))))
+ (exp "-" exp (lambda(args _terminals) (- (float (nth 0 args)) (nth 2
args))))
+ (exp "*" exp (lambda(args _terminals) (* (float (nth 0 args)) (nth 2
args))))
+ (exp "/" exp (lambda(args _terminals) (/ (float (nth 0 args)) (nth 2
args))))
+ ("-" exp %prec NEG (lambda(args _terminals) (- (float (nth 1 args)))))
+ (exp "^" exp (lambda(args _terminals) (expt (float (nth 0 args)) (nth 2
args))))
+ ("(" exp ")" (lambda(args _terminals) (nth 1 args)))))
start))
(parser-generator-process-grammar)
(should-error
@@ -1457,7 +1457,18 @@
(insert "abac")
(parser-generator-set-grammar
- '((Sp S R T) ("a" "b" "c") ((Sp S) (S (R S) (R)) (R ("a" "b" T
(lambda(args) (list "begin" (nth 2 args) "end")))) (T ("a" T (lambda(args)
"test")) ("c") (e))) Sp))
+ '(
+ (Sp S R T)
+ ("a" "b" "c")
+ (
+ (Sp S)
+ (S (R S) (R))
+ (R ("a" "b" T (lambda(args _terminals) (list "begin" (nth 2 args)
"end"))))
+ (T ("a" T (lambda(args _terminals) "test")) ("c") (e)
+ )
+ )
+ Sp)
+ )
(parser-generator-set-look-ahead-number 2)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -1714,8 +1725,8 @@
(
(S (E $))
(E
- (E "*" B (lambda(args) (let ((ret (list (nth 0 args)))) (when (nth 2
args) (setq ret (append ret `(" x " ,(nth 2 args))))) ret)))
- (E "+" B (lambda(args) (let ((ret (list (nth 0 args)))) (when (nth 2
args) (setq ret (append ret `(" . " ,(nth 2 args))))) ret)))
+ (E "*" B (lambda(args _terminals) (let ((ret (list (nth 0 args))))
(when (nth 2 args) (setq ret (append ret `(" x " ,(nth 2 args))))) ret)))
+ (E "+" B (lambda(args _terminals) (let ((ret (list (nth 0 args))))
(when (nth 2 args) (setq ret (append ret `(" . " ,(nth 2 args))))) ret)))
(B)
)
(B
@@ -1765,7 +1776,17 @@
(switch-to-buffer buffer)
(insert "aabb")
- (parser-generator-set-grammar '((Sp S) ("a" "b") ((Sp S) (S (S "a" S "b"
(lambda(args) (let ((list "")) (dolist (item args) (when item (setq list
(format "%s%s" item list)))) list)))) (S e)) Sp))
+ (parser-generator-set-grammar
+ '
+ (
+ (Sp S)
+ ("a" "b")
+ (
+ (Sp S)
+ (S (S "a" S "b" (lambda(args _terminals) (let ((list "")) (dolist (item
args) (when item (setq list (format "%s%s" item list)))) list))))
+ (S e)
+ )
+ Sp))
(parser-generator-set-look-ahead-number 1)
(setq
parser-generator--e-identifier
@@ -1804,7 +1825,18 @@
(switch-to-buffer buffer)
(insert "if (a) { b; }")
- (parser-generator-set-grammar '((Sp S) (";" OPEN_ROUND_BRACKET
CLOSE_ROUND_BRACKET ECHO IF OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE)
((Sp S) (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET VARIABLE ";" CLOSE_CURLY_BRACKET (lambda(args) (format
"(when %s %s)" (nth 2 args) (nth 5 args)))))) Sp))
+ (parser-generator-set-grammar
+ '
+ (
+ (Sp S)
+ (";" OPEN_ROUND_BRACKET CLOSE_ROUND_BRACKET ECHO IF OPEN_CURLY_BRACKET
CLOSE_CURLY_BRACKET VARIABLE)
+ (
+ (Sp S)
+ (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET VARIABLE ";" CLOSE_CURLY_BRACKET (lambda(args _tokens)
(format "(when %s %s)" (nth 2 args) (nth 5 args)))))
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -1857,7 +1889,18 @@
(switch-to-buffer buffer)
(kill-region (point-min) (point-max))
- (parser-generator-set-grammar '((Sp S T) (";" OPEN_ROUND_BRACKET
CLOSE_ROUND_BRACKET ECHO IF OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE)
((Sp S) (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET T CLOSE_CURLY_BRACKET (lambda(args) (format "(when %s %s)"
(nth 2 args) (nth 5 args))))) (T (ECHO VARIABLE ";" (lambda(args) (format
"(message %s)" (nth 1 args)))) (VARIABLE ";" (lambda(args) (format "%s" (nth 0
args)))))) Sp))
+ (parser-generator-set-grammar
+ '(
+ (Sp S T)
+ (";" OPEN_ROUND_BRACKET CLOSE_ROUND_BRACKET ECHO IF OPEN_CURLY_BRACKET
CLOSE_CURLY_BRACKET VARIABLE)
+ (
+ (Sp S)
+ (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET T CLOSE_CURLY_BRACKET (lambda(args _terminals) (format
"(when %s %s)" (nth 2 args) (nth 5 args)))))
+ (T (ECHO VARIABLE ";" (lambda(args _terminals) (format "(message %s)"
(nth 1 args)))) (VARIABLE ";" (lambda(args _terminals) (format "%s" (nth 0
args)))))
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
@@ -1886,7 +1929,17 @@
(switch-to-buffer buffer)
(insert "if (a) { b; }")
- (parser-generator-set-grammar '((Sp S) (";" OPEN_ROUND_BRACKET
CLOSE_ROUND_BRACKET IF OPEN_CURLY_BRACKET CLOSE_CURLY_BRACKET VARIABLE) ((Sp S)
(S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET OPEN_CURLY_BRACKET
VARIABLE ";" CLOSE_CURLY_BRACKET (lambda(args) (format "(when %s %s)" (nth 2
args) (nth 5 args)))))) Sp))
+ (parser-generator-set-grammar
+ '(
+ (Sp S)
+ (";" OPEN_ROUND_BRACKET CLOSE_ROUND_BRACKET IF OPEN_CURLY_BRACKET
CLOSE_CURLY_BRACKET VARIABLE)
+ (
+ (Sp S)
+ (S (IF OPEN_ROUND_BRACKET VARIABLE CLOSE_ROUND_BRACKET
OPEN_CURLY_BRACKET VARIABLE ";" CLOSE_CURLY_BRACKET (lambda(args _terminals)
(format "(when %s %s)" (nth 2 args) (nth 5 args)))))
+ )
+ Sp
+ )
+ )
(parser-generator-set-look-ahead-number 1)
(parser-generator-process-grammar)
(parser-generator-lr-generate-parser-tables)
- [elpa] externals/parser-generator 5cb63eb 399/434: Improve error message of invalid global declaration, (continued)
- [elpa] externals/parser-generator 5cb63eb 399/434: Improve error message of invalid global declaration, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator f9223ea 400/434: Added a flag to use shift conflict resolution in cases were precedence is missing, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator ded7700 387/434: Added more test for infix precedence, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 9e8b89b 392/434: More work on exporting LR-parser, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator fc48311 409/434: Added test for FIRST calculation of multiple symbols and e-identifiers, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 6733b57 415/434: Added feature to move lex-analyzer forward, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 9eca533 410/434: Added test and implementation of calculating FIRST on multiple symbols were first is a non-terminal that expands into an e-identifier, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator ca21bcd 412/434: Shortened doc-string, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 0a3f5f5 426/434: Added test for LR(1) PHP 8.0 match grammar, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator c9906f7 420/434: Move order of export of LR lex-analyzer, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator a4c5d9f 429/434: Syntax-directed translations now have access to terminal data if any is available,
ELPA Syncer <=
- [elpa] externals/parser-generator 428207c 432/434: Fixed bug with exported lr-parser with partial translation, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator fa43f05 301/434: More cpu optimizations, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator a65b652 307/434: Added TODO-items, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator ac7207a 292/434: Passed translate test on exported parser, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 63bd6c0 423/434: LR-parser now has SDT as optional feature to speed up plain parses, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 9c41807 424/434: Code-styling fix, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 231bf85 422/434: Using (signal) instead of (error) in LR-parser and exported LR-parser when encountering syntax error, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator 48f109c 417/434: Fixed typo in exported LR parser, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator da14fbd 431/434: Passing more metadata about terminals to SDT, ELPA Syncer, 2021/11/29
- [elpa] externals/parser-generator af9ad94 418/434: Move lexer flag now affecting global state, ELPA Syncer, 2021/11/29