[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Dynamic token kinds
From: |
Hans Åberg |
Subject: |
Re: Dynamic token kinds |
Date: |
Mon, 17 Dec 2018 11:00:49 +0100 |
> On 17 Dec 2018, at 10:48, Frank Heckenbach <address@hidden> wrote:
>
> Hans Åberg wrote:
>
>>> On 16 Dec 2018, at 15:48, Frank Heckenbach <address@hidden> wrote:
>>>
>>> Hans Åberg wrote:
>>>
>>>> The idea would be that rather than returning the token value,
>>>> something that does not need translation. I don't know if that
>>>> helps up the static approach, though.
>>>
>>> Not sure what you mean here. Please elaborate.
>>
>> I couldn't see the details when I looked at it. I don't use the typed
>> parser, but might if it is safe.
>
> The parser is type safe. This is only about an alternative way of
> creating tokens by the lexer, which is also type safe when used
> properly (as mine does). It's only about adding an additional safety
> net.
Right.
>>> I think Akim made it clear enough that he doesn't like the overhead.
>>> (I don't mind so much, as I used std::variant in my implementation,
>>> but I only have a few parsers to care about.)
>>
>> In that case, my impression was that he thought he could do without it.
>
> Well, he can. :)
In that case.
>>> One might validly say that preventing it is the job of the lexer
>>> (and my lexer does so), not Bison's, just like other kinds of
>>> undefined or wrong behaviour outside of the generated code, also
>>> dynamic token types are a somewhat special usage anyway, so Bison
>>> can just do nothing about it, that's fine.
>>
>> I use the same thing, returning the token value found on a lookup
>> table, but I would not want to use the typed parser if I would
>> have to add translations for every possibility. The information
>> about it is in Bison, therefore it should not be put on the
>> writing of the lexer.
>
> I think we agree here, and that was actually my concern when I
> started this thread. I don't want to have to write a separate case
> for each token kind in my lexer. Of course, we need a separate case
> for each semantic type because that involves a different type in the
> constructor/builder call already, but these are relatively few,
> compared to token kinds, in my lexers.
Might Bison generate a function with a switch statement, generate the right
return for the lexer to use?
>>> I also suggested an approach in my previous mail with a few more
>>> generated functions that help runtime checking. I'd prefer Bison to
>>> add them, and then we'd have runtime checking as good as we'd have
>>> with std::variant in this respect.
>>
>> Maybe an option. Akim perhaps haven't used this dynamic token
>> lookup.
>
> I guess he hasn't. But I don't think we need an option. These would
> just be additional functions that one can use or not.
The with an option would be that those that do not need this feature could use
a more optimal variant.
>> Those that do might prefer not risking the program to bomb.
>
> It's not that bad actually. Again, my lexers work fine as is.
> I just brought this up because Akim proposed to call the function
> "unsafe_..." which I thought was too harsh and proposed
> "unchecked_..." -- but by adding the checks, it would be neither
> unsafe nor unchecked. :)
This worries me. But also having having to use something more complex to be
returned by the lexer than a value on the lookup table .
- Re: Dynamic token kinds, Hans Åberg, 2018/12/15
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/16
- Re: Dynamic token kinds, Hans Åberg, 2018/12/16
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/16
- Re: Dynamic token kinds, Hans Åberg, 2018/12/16
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/16
- Re: Dynamic token kinds, Hans Åberg, 2018/12/16
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/17
- Re: Dynamic token kinds,
Hans Åberg <=
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/17
- Re: Dynamic token kinds, Hans Åberg, 2018/12/17
- Re: Dynamic token kinds, Frank Heckenbach, 2018/12/17
- Re: Dynamic token kinds, Hans Åberg, 2018/12/17
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/17
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/17
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/18
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/18
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Frank Heckenbach, 2018/12/19
- Re: Porting to typed C++ parser (was: Dynamic token kinds), Hans Åberg, 2018/12/19