[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Parsing included files, part II.
From: |
Ervin Hegedüs |
Subject: |
Parsing included files, part II. |
Date: |
Sun, 23 Aug 2020 20:34:37 +0200 |
User-agent: |
NeoMutt/20171215 |
Hi all,
(sorry, a bit long and difficult description...)
few days ago I asked a question about config file inclusions:
https://lists.gnu.org/archive/html/help-bison/2020-08/msg00007.html
and I wrote that I solved that:
https://lists.gnu.org/archive/html/help-bison/2020-08/msg00012.html
I rewrited the code, now I can include files on unlimited levels,
because I switched to dynamic memory handling. Everíthing works
as well - if I use the "include" directive sequentially.
Now I'ld like to move on, and try to implement the "include
patterns", eg.:
include foo*.conf
Here are the relevant part of my code:
in my lexer:
============
<INITIAL>include {
BEGIN(ST_INCLUDE_DIRECTIVE); push_sym(); return T_INCLUDE_DIRECTIVE; }
<ST_INCLUDE_DIRECTIVE>[0-9A-Za-z_\/\.\-\*\:]+ { include_stack_ptr++;
push_sym(); parse_config(yytext); BEGIN(INITIAL); return
T_INCLUDE_DIRECTIVE_ARGUMENT; }
<ST_INCLUDE_DIRECTIVE>\"[0-9A-Za-z_\/\.\-\*\:]+\" { include_stack_ptr++;
push_sym(); parse_config(yytext); BEGIN(INITIAL); return
T_INCLUDE_DIRECTIVE_ARGUMENT_QUOTED; }
<<EOF>> {
printf("Got EOF, stack_ptr: %d\n", include_stack_ptr);
BEGIN(INITIAL);
if (include_stack_ptr >= 0) {
close_and_cleanup(0);
} else {
yyterminate();
}
}
my parser:
==========
config_include:
T_INCLUDE_DIRECTIVE T_INCLUDE_DIRECTIVE_ARGUMENT { printf("%s
%s\n", $1, $2); free($1); free($2); }
| T_INCLUDE_DIRECTIVE T_INCLUDE_DIRECTIVE_ARGUMENT_QUOTED { printf("%s
'\"%s\"'\n", $1, $2); free($1); free($2); }
;
notes:
* push_sym() just push the recognized token to stack, so I can
use that in parser:
yylval.s = strdup(yytext);
* include_stack_ptr points to top of different stacks, eg. files,
buffers, ...
* parse_config() gets the filename, expands it to full path, and
tries to handle it with wordexp(3), like this:
wordexp(exp_fullname, &p, 0);
w = p.we_wordv;
for (size_t wi = 0; wi < p.we_wordc; wi++) {
... few error handling...
open_and_parse(...)
* the open_and_parse() tries to open the file, makes some
modifications into inputbuff (a local buffer), and starts to
parse:
if (include_stack_ptr == 0) {
yy_scan_string(inputbuff);
yyparse();
}
else {
include_stack[include_stack_ptr-1] = YY_CURRENT_BUFFER;
yy_scan_string(inputbuff);
}
* there is the close_and_cleanup(), which closes the opened
files, cleans up the not-used stack items, and so on...
This routine is called when <<EOF>> detected.
Here are two examples:
$ ls -1 testinc/
foo1.conf
foo2.conf
incfoo1.conf
incfoo2.conf
$ cat testinc/foo1.conf
# file: foo1.conf
$ cat testinc/foo2.conf
# file: foo2.conf
$ cat testinc/incfoo1.conf
include foo1.conf
include foo2.conf
and finally:
$ cat testinc/incfoo2.conf
include foo*.conf
So, my code works when I try to parse the incfoo1.conf, which
contains the included files sequentially:
Opening file: /home/airween/src/parsertest/testinc/incfoo1.conf
Opening file: /home/airween/src/parsertest/testinc/foo1.conf
include foo1.conf <- this lines is created by the parser, see below
THIS IS A COMMENT: '# file: foo1.conf' <- this is the content of file
Got EOF, stack_ptr: 1 <- IMPORTANT, here is the EOF
Closing file: /home/airween/src/parsertest/testinc/foo1.conf, stack_ptr: 1
Opening file: /home/airween/src/parsertest/testinc/foo2.conf
include foo2.conf <- also by the parser
THIS IS A COMMENT: '# file: foo2.conf'
Got EOF, stack_ptr: 1
Closing file: /home/airween/src/parsertest/testinc/foo2.conf, stack_ptr: 1
Got EOF, stack_ptr: 0 <- end of incfoo1.conf
Closing file: /home/airween/src/parsertest/testinc/incfoo1.conf, stack_ptr: 0
Parser finished
Now here is the output when I call it with incfoo2.conf, which
contains "include foo*.conf":
Opening file: /home/airween/src/parsertest/testinc/incfoo2.conf
Opening file: /home/airween/src/parsertest/testinc/foo1.conf
Opening file: /home/airween/src/parsertest/testinc/foo2.conf <- wait, where is
the EOF?
include foo*.conf <- here is the output of parser
THIS IS A COMMENT: '# file: foo2.conf' <- uhm, this is the last file
Got EOF, stack_ptr: 1 <- oh, here is the EOF of last file
Closing file: /home/airween/src/parsertest/testinc/foo2.conf, stack_ptr: 1
THIS IS A COMMENT: '# file: foo1.conf' <- ops, this is the *first* file
Got EOF, stack_ptr: 0 <- ahm, and this is the first EOF
Closing file: /home/airween/src/parsertest/testinc/incfoo2.conf, stack_ptr: 0
as you can see:
* the order of files are reversed
* at least one file closing is missing (because one EOF missing)
Do you have any idea, why can't realize the lexer the <<EOF>> of
first included file? And why had changed the order of files?
I know this is a very complicated code and description, but I
hope somebody made a similar code, and can help me :).
Thank you,
a.
- Parsing included files, part II.,
Ervin Hegedüs <=