Re: How to free symbols during error recovery

[Akim Demaille]

>>> "Laurence" == Laurence Finston <address@hidden> writes:

 > Would it be possible for someone to put the maual for release 1.875
 > there instead?

Lack of time, but will be done, thanks.

 > According to my understanding, this feature makes it possible for a
 > function to be called that destroys objects referenced by symbols
 > in the parser rules when states are popped from the stack.  This is
 > an excerpt from the section about error recovery in the Bison
 > manual for Release 1.35:

 >    But Bison can force the situation to fit the rule, by discarding
 >    part of the semantic context and part of the input. First it
 >    discards states and objects from the stack until it gets back to
 >    a state in which the error token is acceptable. (This means that
 >    the subexpressions already parsed are discarded, back to the
 >    last complete stmnts.) At this point the error token can be
 >    shifted. Then, if the old look-ahead token is not acceptable to
 >    be shifted next, the parser reads tokens and discards them until
 >    it finds a token which is acceptable. In this example, Bison
 >    reads and discards input until the next newline so that the
 >    fourth rule can apply.

 > I've assumed that this causes the destructor to be called.
 > Is this correct?

No, rather it says when we stop throwing away symbols, hence error
"stops" %destructor.  It does not trigger it.

 > I haven't considered using `%destructor' for my application,
 > firstly because I haven't gotten around to it, and secondly because
 > I don't think it will work for my parser.  I use a `%union'
 > containing a `void*' that can point to objects of various types
 > that are not all in a single class hierarchy, so writing a
 > destructor function would be complicated.  

I don't understand the relationship here.

 > Nor do I want want states and objects to be discarded in a way
 > that's not under my control.  I also want to determine the position
 > from which parsing is resumed.

I have not understood how you do that.  You can't unless you changed
the parser.

 > I have an idea for a solution, and if anyone's interested I will be
 > happy to explain it.  On the other hand, if it turns out that
 > `%destructor' will do the job more simply, I'll happily use it
 > instead.

Here is the (up to date) doc.  (The doc says the syntax might change,
but I no longer believe so).

----------------------------------------------------------------------

3.7.6 Freeing Discarded Symbols
-------------------------------

Some symbols can be discarded by the parser.  For instance, during error
recovery (*note Error Recovery::), embarrassing symbols already pushed
on the stack, and embarrassing tokens coming from the rest of the file
are thrown away until the parser falls on its feet.  If these symbols
convey heap based information, this memory is lost.  While this behavior
can be tolerable for batch parsers, such as in compilers, it is not for
possibly "never ending" parsers such as shells, or implementations of
communication protocols.

   The `%destructor' directive allows for the definition of code that
is called when a symbol is thrown away.

 -- Directive: %destructor { CODE } SYMBOLS
     Declare that the CODE must be invoked for each of the SYMBOLS that
     will be discarded by the parser.  The CODE should use `$$' to
     designate the semantic value associated to the SYMBOLS.  The
     additional parser parameters are also available (*note The Parser
     Function `yyparse': Parser Function.).

     *Warning:* as of Bison 1.875, this feature is still considered as
     experimental, as there was not enough user feedback.  In
     particular, the syntax might still change.

   For instance:

     %union
     {
       char *string;
     }
     %token <string> STRING
     %type  <string> string
     %destructor { free ($$); } STRING string

guarantees that when a `STRING' or a `string' will be discarded, its
associated memory will be freed.

   Note that in the future, Bison might also consider that right hand
side members that are not mentioned in the action can be destroyed.  For
instance, in:

     comment: "/*" STRING "*/";

the parser is entitled to destroy the semantic value of the `string'.
Of course, this will not apply to the default action; compare:

     typeless: string;  // $$ = $1 does not apply; $1 is destroyed.
     typefull: string;  // $$ = $1 applies, $1 is not destroyed.


   "Discarded symbols" are the following:

   * stacked symbols popped during the first phase of error recovery,

   * incoming terminals during the second phase of error recovery,

   * the current look-ahead when the parser aborts (either via an
     explicit call to `YYABORT', or as a consequence of a failed error
     recovery).