[Devel] Summary of ANSI preprocessor trouble..

[David Turner]

Hello all,

Here's a short summary of the situation regarding the infamous
pre-processor concatenation problem. I hope it clarify things..

> 
> Apparently, macros are not expanded for some compilers before
> concatenation.  For example,
> 
>   #define FOO xxx
>   #define BAR FOO ## yyy
> 
> gives `FOOyyy' instead of `xxxyyy'.
> 

All I know for sure is what in my K&R ANSI C book (second edition)
It basically states the following regarding the preprocessor:

  - lexems are formed by _any_ suite of contiguous characters
    that are not whitespace !! So #include <FOO/xxxx> should normally
    never substitute, even though it does in a number of cases..

  - when either '#' or '##' is used, macro substitution of the
    arguments does _not_ happen. This explains the "FOOyyy"
    thing

  - when lexem concatenation is used, the whitespace surrounding the "##"
    is discarded. which means that

       #define  z x ## y

    will result in "z xy"

    A FEW ANSI C COMPILERS DO NOT RESPECT THIS RULE. To my knowledge, these
    are "Win32-LCC" (which comes with its own buggy CPP), and a beta version
    of gcc (I believe that the C pre-processor has been completely rewritten
    as a library for the upcoming gcc 3.0, maybe this explains the bugs)

    this means that #define  z x ## y gives "z x y" or something similar
    with these..

    I suppose we could provide a special version of <ft2build.h>
    for them if they really need to


  - a common way to perform concatenation with macro substitution is to use
    a two-level scheme like (according to K&R):

      #define  FT_XCAT(x,y)  x ## y
      #define  FT_CAT(x,y)   FT_XCAT(x,y)

    then

      #define  FOO   foo
      #define  BAR   FT_CAT(FOO,bar)

    will set BAR to "foobar", because FOO is substituted into "foo" when
    parsing the definition of FT_CAT, not in FT_XCAT.


   - to make things worse, Werner recently sent me this piece of poetry
     from the GNU cpp.info:

  """
   The usual case of concatenation is concatenating two names (or a
   name and a number) into a longer name.  But this isn't the only
   valid case.  It is also possible to concatenate two numbers (or a
   number and a name, such as `1.5' and `e3') into a number.  Also,
   multi-character operators such as `+=' can be formed by
   concatenation.  In some cases it is even possible to piece together
   a string constant.  However, two pieces of text that don't together
   form a valid lexical unit cannot be concatenated.  For example,
   concatenation with `x' on one side and `+' on the other is not
   meaningful because those two characters can't fit together in any
   lexical unit of C.  The ANSI standard says that such attempts at
   concatenation are undefined, but in the GNU C preprocessor it is
   well defined: it puts the `x' and `+' side by side with no
   particular special results.
  """

This clearly contradicts Kernighan and Richie !! It seems we need to
put our hands on the _real_ ANSI C standard and see what the exact words
are on it.. Anyone has this document and could share a quick lecture
with us ??

Thanks,

- David