Re: [Texmacs-dev] Recent Works on Programming Language Parsers

[Darcy Shen]

Well, I remember that there are bugs. But I failed to reproduce it today.

Maybe, I was using GNU TeXmacs 1.99.12. I should have submitted bug report.

On 2020/3/30 下午10:06, Darcy Shen via Texmacs-dev wrote:
> > OK, for the next several months, only bug fixes will be committed.
>
> Unfortunately, there are bugs from
> `Language(C++): re-impl coloring using improved string_parser`
> to
> `Language(DOT): tuning on dot-edit`.
>
> I will fix it next weekend and add the missing test cases.
>
> On 2020/3/29 上午9:43, TeXmacs wrote:
> > Thanks Darcy.  With Covid-19 raging, it is indeed a top priority to
> > focus on the community.  You are also in an ideal position for guiding
> > Chinese students who may wish to help in the GSoC context.
> > I am extremely busy, but hope to contribute a few ideas as well later.
> >
> > Best wishes, --Joris
> >
> >
> > On Sat, Mar 28, 2020 at 03:26:03PM +0800, Darcy Shen via Texmacs-dev 
> > wrote:
> > > > In the next one or two months, I will continue to work on improving 
> > > > the xyz_parser and abc_language.
> > > OK, for the next several months, only bug fixes will be committed.
> > >
> > > And last week, we have some good news on GSoC. For the next serveral 
> > > months, I will focus more on the community:
> > >
> > > 1. Write documentation on devel (https://github.com/texmacs/devel) 
> > > for new contributors
> > > 2. Review pull requests
> > > 3. ...
> > >
> > > On 2020/3/26 ??????9:07, TeXmacs wrote:
> > >
> > > > Hi Darcy,
> > > >
> > > > I noticed your recent changes.
> > > > Please be very careful when commiting changes,
> > > > since version 2.1 is still approaching, despite the Covid-19 setback.
> > > >
> > > > Best wishes, --Joris
> > > >
> > > >
> > > > On Mon, Mar 23, 2020 at 02:36:03AM +0800, Darcy Shen via 
> > > > Texmacs-dev wrote:
> > > > > Recent tree months, I wrote some parsers under src/Data/Parser:
> > > > >
> > > > > + blanks_parser
> > > > >
> > > > > + escaped_char_parser
> > > > >
> > > > > + identifier_parser
> > > > >
> > > > > + inline_comment_parser
> > > > >
> > > > > + keyword_parser
> > > > >
> > > > > + number_parser
> > > > >
> > > > > + operator_parser
> > > > >
> > > > > + string_parser (TODO: multi-line support)
> > > > >
> > > > > On top of these parsers, I rewrite various part of xxx_language.cpp .
> > > > >
> > > > > I didn't use a higher-level abstraction like packrat-parser. In my
> > > > > opinion,for now, low level small parsers written in C++ should work
> > > > > fine.
> > > > >
> > > > > Most programing languages are similar in syntax. Doing simple
> > > > > abstraction is sufficient for syntax highlighting.
> > > > >
> > > > >
> > > > > The goal of these small parsers with `can_parser` and `do_parser` is
> > > > > to minimize the time to implemented a new programming language
> > > > > parser.
> > > > >
> > > > >
> > > > >       Below are the detailed progress:
> > > > >
> > > > > + dot: new composition of parsers, string_parser, keyword_parser,
> > > > > operator_parser
> > > > >
> > > > > + cpp: new composition of parsers, string_parser
> > > > >
> > > > > + java/scala/python: old composition of parsers, string_parser
> > > > >
> > > > > + others: old composition of parsers
> > > > >
> > > > >
> > > > >       Composition of parsers: old vs new
> > > > >
> > > > > Almost all xxx_language.cpp (except scheme_language.cpp) is derived
> > > > >from mathemagix_language.cpp .
> > > > > I call it old-style composition of parsers. It is not efficient
> > > > > enough, because, we have to re-parse the code in `get_color`.
> > > > >
> > > > > Dive into concat_text.cpp:typeset_prog_string, we will find what
> > > > > actually is `get_color` and `advance`.
> > > > >
> > > > > The new-style composition of parsers, reduce the unnecessary
> > > > > parsings in get_color.
> > > > >
> > > > >
> > > > >       Keyword/Operator Parser
> > > > >
> > > > > Aims to keep the (type,  keyworkd) mapping in Scheme files. Please
> > > > > refer to `dot-lang.scm`.
> > > > >
> > > > >
> > > > >       String Parser
> > > > >
> > > > > Currently, the String parser only support inline string. Actually,
> > > > > string and multi-comment are the same type.
> > > > >
> > > > > They both have openings and corresponding closings.
> > > > >
> > > > > The string parser will finally support multi-line. Once it is ready,
> > > > > the multi-comment parser will also be implemented
> > > > >
> > > > > in a short time.
> > > > >
> > > > >
> > > > >       Recent Plans
> > > > >
> > > > > I will continue my developments on newly-supported languages (like
> > > > > dot). The goal is to make it extremely easy to
> > > > >
> > > > > support a new language. For coloring schemes, it is another topic.
> > > > > In the next one or two months, I will continue to
> > > > >
> > > > > work on improving the xyz_parser and abc_language.
> > > > >
> > > > >
> > > > > Darcy
> > > > >
> > > > > 2020/03/23
> > > > >
> > > > > _______________________________________________
> > > > > Texmacs-dev mailing list
> > > > > address@hidden
> > > > > https://lists.gnu.org/mailman/listinfo/texmacs-dev
> > > > _______________________________________________
> > > > Texmacs-dev mailing list
> > > > address@hidden
> > > > https://lists.gnu.org/mailman/listinfo/texmacs-dev
> > >
> > > _______________________________________________
> > > Texmacs-dev mailing list
> > > address@hidden
> > > https://lists.gnu.org/mailman/listinfo/texmacs-dev
> > _______________________________________________
> > Texmacs-dev mailing list
> > address@hidden
> > https://lists.gnu.org/mailman/listinfo/texmacs-dev
>
>
> _______________________________________________
> Texmacs-dev mailing list
> address@hidden
> https://lists.gnu.org/mailman/listinfo/texmacs-dev