[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Tokenizing
From: |
Vladimir Kazanov |
Subject: |
Re: Tokenizing |
Date: |
Sat, 20 Sep 2014 19:36:16 +0300 |
Okay, I'll give text properties a try.
Right now my vision for this mode is the following:
- avoid retokenizing undamaged buffer parts at all costs (as a main
feature meant for incremental parsing);
- collect damages and do reparsing only when user stops editing,
similar to the font-lock-mode (js2-mode, nxml-mode...);
- the incremental logic should have two interfaces, the first one
meant for language-specific tokenizing code and a second one - for the
user code, be it code beautifiers or advanced incremental parsers;
- it should be possible to completely replace the font-lock-mode with
this mode, given a concrete language tokenizer;
You said two things basically: 1) I must use text properties, 2) it is
possible to improve text properties interfaces to help the tokenizer.
I suggest the following plan:
1) try to implement the tokenizer using available text property mechanics;
2) see if there are slow-downs or problems, or space for improvements
on the Emacs side.
Does that sound reasonable?
On Sat, Sep 20, 2014 at 4:21 PM, Richard Stallman <address@hidden> wrote:
> [[[ To any NSA and FBI agents reading my email: please consider ]]]
> [[[ whether defending the US Constitution against all enemies, ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Do you plan to reparse continuously to keep the token structure up to
> date all the time?
>
> Given that reparsing is likely to take time, it may not be the best approach.
> It could sometimes make Emacs become nonresponsive while it is reparsing
> a substantial amount of text.
>
> Ideally it should be like Font Lock -- reparsing parts of the buffer
> when time is available.
>
> --
> Dr Richard Stallman
> President, Free Software Foundation
> 51 Franklin St
> Boston MA 02110
> USA
> www.fsf.org www.gnu.org
> Skype: No way! That's nonfree (freedom-denying) software.
> Use Ekiga or an ordinary phone call.
>
--
Yours sincerely,
Vladimir Kazanov
--
С уважением,
Владимир Казанов
- Re: Overlay mechanic improvements, (continued)
Re: Overlay mechanic improvements, Richard Stallman, 2014/09/19
- Re: Overlay mechanic improvements, Vladimir Kazanov, 2014/09/20
- Tokenizing, Richard Stallman, 2014/09/20
- Re: Tokenizing, Stephen Leake, 2014/09/20
- Re: Tokenizing, Vladimir Kazanov, 2014/09/20
- Re: Tokenizing, Eric Ludlam, 2014/09/20
- Re: Tokenizing, Vladimir Kazanov, 2014/09/20
- Re: parsing (was tokenizing), Stephen Leake, 2014/09/21
Re: Tokenizing,
Vladimir Kazanov <=
Re: Tokenizing, Stefan Monnier, 2014/09/20
Re: Tokenizing, Stephen Leake, 2014/09/21
Re: Tokenizing, Stefan Monnier, 2014/09/21
Re: Tokenizing, Stephen Leake, 2014/09/22
Re: Tokenizing, Richard Stallman, 2014/09/21
Re: Tokenizing, Vladimir Kazanov, 2014/09/21
Re: Tokenizing, Stephen Leake, 2014/09/21
Re: Tokenizing, Stefan Monnier, 2014/09/21
Re: Tokenizing, Vladimir Kazanov, 2014/09/21
Re: Tokenizing, Daniel Colascione, 2014/09/21