[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Which lexer do people use?

From: Akim Demaille
Subject: Re: Which lexer do people use?
Date: Mon, 6 Jul 2020 18:03:07 +0200

> Le 6 juil. 2020 à 14:00, Maury Markowitz <> a écrit :
>> On Jul 4, 2020, at 6:46 AM, Christian Schoenebeck 
>> <> wrote:
>> The only thing that people are missing once in a while on scanner side is 
>> unicode support
> I am missing the capability to support grammars with optional spaces between 
> elements. In BASIC:
>   FORI=1TO5
> is equivalent to:
>   FOR I=1 TO 5
> I have yet to see an elegant way to model this in Flex. This topic comes up 
> time to time here, so I don't think Unicode is the "only" thing.

I believe this is a perfect place to use Flex's / operator I mentioned
(similar to what other engines call "lookahead", and write 'r(?=s)').

     an 'r' but only if it is followed by an 's'.  The text matched by
     's' is included when determining whether this rule is the longest
     match, but is then returned to the input before the action is
     executed.  So the action only sees the text matched by 'r'.  This
     type of pattern is called "trailing context".  (There are some
     combinations of 'r/s' that flex cannot match correctly.  *Note
     Limitations::, regarding dangerous trailing context.)

reply via email to

[Prev in Thread] Current Thread [Next in Thread]