News of my porting nightmare

[Hans-Bernhard Broeker]

(This is a followup to my earlier posting...)

I managed to solve most, maybe all of the mysteries myself, in the
meantime. The key problem turned out to be that the given scanner was
using yyless() and yymore() in the same action, repeatedly. (Some of those
usages I introduced myself as I tried to speed it up for flex). It took me
several readings of the manual and the generated C source to realize that
yyless() modifies yyleng, too, and thus is not allowed in the same action
with yymore(). I'd love to see the manual changed to point this out a bit
more directly.

I ended up writing my own version of yymore() that copies the matched text
to another array, my_yytext, instead of keeping it in yytext.

I also removed the input() routine override, and let flex do the
comment-parsing, instead. This turned out to be a major performance
enhancement. Looks like the real bootleneck was that the home-grown
input() only returned a single character at a time, and it was using
single getc()s to read from yyin. It turned out that getc() was called so
often that its shared library call overhead alone consumed a big part of
the runtime --- linking the whole program statically made it 30% faster

The resulting scanner is now within a factor of 2 to the speed of an
AT&T-lex generated one, I think.

Hans-Bernhard Broeker (address@hidden)
Even if all the snow were burnt, ashes would remain.