[Gomp-discuss] Lexing and parsing OMP directives, again

[Steven Bosscher]

Zack Weinberg wrote in http://gcc.gnu.org/ml/gcc/2003-02/msg00653.html:
> Steven Bosscher <address@hidden> writes:
> > 
> > Pragmas are registered in cpplib with a handler that is called from
> > cpplib.  So the lexer and parser never see the pragmas. This makes it a
> > bit difficult to have elaborate interaction between the parser and the
> > pragma handlers.  There are additional callbacks for most pragmas from
> > c-decl (the maybe_apply_{weak,renaming_pragma} functions) because the
> > compiler may encounted symbols that are affected by those pragmas long
> > after the pragma itself was handled.
> >
> > In OpenMP, the pragmas are more like grammar productions.  Certain
> > pragmas can only be followed or by a compound statement, some other
> > pragmas can only appear inside a compound.  Some constructs need
> > information about scope and/or about how deep they're nested.
> >
> > In fact, the OpenMP specifications present the OpenMP pragmas as grammar
> > extensions for C/C++!
> 
> Pragmas are callbacks only for historical reasons: It was easier to
> implement them that way at the time.  With the C/C++ front ends moving
> to recursive descent parsers, it would make perfect sense to put them
> back in the token stream.  Perhaps we would transform
> 
> #pragma foo bar baz
> 
> into
> 
> __builtin_pragma foo bar baz;
> 
> and feed that to phase 7.
> 
> zw


Hi,

With the GOMP project we still haven't found out how to lex/parse OMP
directives.  That bothers me a bit so I've been giving this a lot of
thought lately, but I still don't see how it can be done without major
changes.  We need to figure this out before we can hack the front ends. 
I was hoping maybe you can give me some help here because you probably
have the best idea of what exactly you mean with "phase 7" and you know
cpplib best :-)

So what problems are we facing when lexing/parsing OpenMP pragmas?  Most
of these have to do with how GCC handles pragmas.  The more I looked at
it, the more I have to agree with the responses we received from the GCC
mailing list last time the GOMP project was discussed there: Pragma's
really suck big time.

AFAICU, cpplib allows the front end to register a "pragma handler" which
can lex tokens with cpplib (with no macro expansion).  The handler
receives CPP_EOF when the preprocessor finds an end-of-line.

_The_ single most important problem we're facing is that GCC can only
handle really simple pragmas that can appear anywhere, ie. have no
syntactic context.  On the other hand, OMP pragmas can only appear in
certain contexts, they can be nested, and they sometimes restrict the
syntax of the expressions following the pragma (eg. "parallel for"). 
This means that we have to figure out a way to make the parser aware of
OpenMP. 

A related problem is that it is almost impossible to implement the "if",
"num_threads" and "reduction" clauses.  These clauses can have full
scalar C/C++ expressions as their arguments.  Calling (expression)
parser from a GCC pragma handler is difficult to do because such
handlers are unaware of the state of the parser.  And obviously it's
even impossible if you're using a YACC-derived parser...  Writing an
expression parser just for OpenMP is _not_ an option, so again, we need
to hook up OMP pragmas with the parser somehow.

Sebastian Pop has been playing with rewriting those "#pragma OMP"s as
builtin _functions_.  He wrote in a message to the GOMP list:
---------------------------
Following the discussion from gcc@ we should transform the entire line
into something that looks like a function call:
#pragma omp flush (x)
to
__builtin_omp_flush (x);

But then what happens when we're given something like that:
#pragma omp parallel shared (j, sum) private (i) schedule (runtime)

Would it be transformed into 
__builtin_omp_parallel (shared (j, sum), private (i), schedule (runtime));
?
---------------------------

I believe it is obvious that this approach is not going to work.  It is
unclear to me how we can handle the syntactic restrictions on
expressions following the OMP pragmas, or nested pragmas, etc. when we
have to handle them as builtin functions.  


After re-reading that mail from you to gcc@, I think you had something
else in mind: Not builtin _functions_, but rather some kind of token
__builtin_pragma, and a regular stream of tokens following that.  If I
understand your idea correctly, this means that the parser would be
(more) responsible for handling pragmas.  This would be ideal to us. 

However, I and probably most others in the GOMP project, are not exactly
C/C++ language lawyers.  At least I don't have a clue what "phase 7"
means, and how it is related to recursive descent parsing. 

Could you please explain your idea a bit further, and maybe point us to
some "Further reading" material?

Gr.
Steven