AW: [Grammatica-users] Tokenizer problem

[HECKHAUSEN Ralf]

Yes, it helps in this example. Let's take a more real example:
%header%
GRAMMARTYPE = "LL"
%tokens%
LETTER = <<[A-Z]>> 
DIGIT = <<[0-9]>>
V = "V"
%productions%
INPUT = STATION_ID VARIATION;
STATION_ID = LETTER LETTER LETTER LETTER;
VARIATION = DIGIT DIGIT DIGIT V DIGIT DIGIT DIGIT;
 
Output for parsing "LOVW 123V345":
 
INPUT(2001)
  STATION_ID(2002)
    LETTER(1001): "L", line: 1, col: 1
    LETTER(1001): "O", line: 1, col: 2
Error: in test: line 1:
    unexpected token "V", expected <LETTER>
LOVW 123V345
  ^
 
What can I do about this?
 
Ralf

 
________________________________

Von: address@hidden im
Auftrag von Anant Mistry
Gesendet: Fr 01/07/2005 14:13
An: address@hidden
Betreff: Re: [Grammatica-users] Tokenizer problem



Ralf,

A possible solution would be to define 

COMMAND = "GO "

That is a string of "GO" followed by the space character.
It makes your production

INPUT = COMMAND LETTER+;

Hope this helps

Anant

On Fri, 2005-07-01 at 10:48 +0200, HECKHAUSEN Ralf wrote:


        I have a problem with the parser, or better the tokenizer. Below is a
simplified example.
        
        %header%
        
        GRAMMARTYPE = "LL"
        
        %tokens%
        LETTER = <<[A-Z]>>
        COMMAND = "GO"
        SPACE = " "
        
        %productions%
        
        INPUT = COMMAND SPACE LETTER+;
        
        
        

        The input "GO SOUTH" is parsed correctly, but with "GO SOGO" I get a
parse error, because "GO" is not recognized as two letters anymore. It seems
that whenever letters are defined as tokens, they may not occur in other
context. But I am sure there is a way around this problem, can someone help
me? 

        

        Cheers, 

        Ralf 

        _______________________________________________
        Grammatica-users mailing list
        address@hidden
        http://lists.nongnu.org/mailman/listinfo/grammatica-users

-- 
Anant Mistry
Director of Infrastructure Technology
Information Handling Services
15 Inverness Way East
Englewood
CO 80112
USA
303 397 2569
 
Confidentiality Notice: The information in this e-mail may be confidential
and / or privileged. This e-mail is intended to be reviewed by only the
individual or organization named in the e-mail address. If you are not the
intended recipient, you are hereby notified that any review,
dissemination or copying of this e-mail and attachments, if any, or the
information contained herein, is strictly prohibited.

<<winmail.dat>>