yyclearin does not invoke the lookahead token's %destructor

[Rici Lake]

(Collected from a StackOverflow question; see
https://stackoverflow.com/questions/48672697/why-does-the-use-of-yyclearin-not-trigger-destructor-in-bison
)

As indicated in the question and answer, yyclearin simply marks the
lookahead token as YYEMPTY. Its associated value is not destructed, and
will (presumably) be overwritten by the next token read, creating a memory
leak.

yyclearin is not in the list of things which trigger a destructor, so the
documentation is not incorrect. However, the summary statement "As a rule
of thumb, destructors are invoked only when user actions cannot manage the
memory." is a bit misleading, since there is no easy way for a user action
to manage the memory of the lookahead token. True, it is *possible* -- the
user action could reproduce the entire %destructor selection logic, or it
could use the crude workaround I suggest in the SO answer (which involves
undocumented bison internals) -- but neither of those seem really
appropriate.

Modifying yyclearin so that it calls yydestruct seems like the simplest
solution to this issue, but it is conceivable that such a change would
break programs which already perform some kind of workaround in order to
destruct the lookahead symbol. So it might be necessary to use some kind of
compatibility %define, or to create a new replacement macro with a
different name such as yydiscardin.

At a minimum, the fact that yyclearin does not invoke the %destructor
should be highlighted in the documentation, since it is not at all obvious.