bug#15483: POSIXLY_CORRECT documentation vis a vis some simple EREs

[Glenn Golden]

--
Eric Blake <address@hidden> [2013-10-07 11:05:09 -0600]:
> 
> So what?  No one cares whether grep is a 'conforming application',
>

Disagree strongly that "no one" cares.

>
> Our counterargument is that we KNOW that grep in not a conforming application,
> and WE DON'T CARE.
>

>From your perspective as developers/maintainers, I understand that _you_ don't
care -- nor should you -- whether grep is a conforming application, or even
an application at all for that matter: Grep was not _designed_ to be an
application within the POSIX framework;  you're not _viewing_ it as an
application; and you're not _maintaining_ it as an application. It's a system
component, and as such, part of a POSIX 'implementation', and it is entirely
conforming as such.  No argument. I agree and support that point of view
entirely. (And btw, the doc patch draft that I had planned to submit also
makes your points very clearly and explicitly, so as to leave no doubt in
the readers' mind that simply because grep is not a conforming application
does not imply that there is any "defect" in grep.)  Totally agree with your
point of view on that.

But!

I completely disagree that _no one_ cares.  As you said of yourself, you're
very close to all this and things which appear obvious to you as maintainers
may not be so obvious to others outside that realm. I claim that users in the
position of writing applications which use grep as a building block certainly
_do_ care about whether grep is or is not a conforming application, and
therefore this "duality" should certainly be called out in the doc, because
it has real relevance in the real world.

That's the case I was in the process of making in the prior post, but wanted
to find out first whether you agreed or not that grep was a non-conforming
application. It matters.

> 
> You're arguing about grep being non-conforming as if it mattered, but
> our counterargument is that it does not matter.
> 

To you it does not. To users writing applications and concerned about POSIX
conformance, I believe it does. I'll continue making that case if you wish,
but now I'm so sure you wish. :) Let me know.

>
> POSIX is perfectly clear that implementations may provide extensions that
> give non-error meaning to any regex that is not strictly defined in the
> narrow subset required by POSIX.
>

I agree entirely.  But the key word above is 'implementations'. It is not
clear that it is true for 'conforming applications' in all cases.

To me, the language contrast in 9.5.3. between "permitted" (pertaining to
'implementations') vs. "cannot use" (pertaining to 'conforming applications')
strongly suggests that in this section of the spec they specifically intended
to distinguish between "behavior which is permitted by internal components of
a POSIX system" [i.e. 'implementations', which certainly may use extended EREs 
and/or diagnose them or not as they see fit] vs. "behavior which is permitted
by conforming applications" [and hence does not fall under the rubric of
'implementation', and which "cannot use" ERE's > P.]  To me, the very presence
of the two distinct sentences is a red flag that they were trying to draw the
readers' attention to a distinction in conformance behavior between the two
entities ('implementations' and 'conforming applications') in regard to EREs.

Just got your follow-up email as I was writing this, which talks about the
above, and suggests bringing it up on the POSIX list. I will do so and spare
you any further noise until I get some response there.

Let me just conclude here by saying that I will be honestly surprised to find
that that they intended the phrase "cannot use x" (for x > P) to include the
interpretation "you may use x if you want, since such x is outside the scope
of POSIX to begin with." To me, it seems peculiar that they would write
"cannot use" so specifically, and it also seems like circular reasoning. If it
is true as you say that

    "...the intent of POSIX is that if an application is presented x > P as
     input, then the application is no longer bound by the rules of POSIX,
     and undefined behavior will occur."

then why was the "cannot use" sentence specifically placed into that section?

By your explanation above, there is no need for that sentence, because for
x > P, *anything* the standard says about an application's behavior is
irrelevant; after all, "the application is no longer bound by the rules of
POSIX".  So why in the world put an explicit sentence into a standard which
constrains an application's behavior, but applies *only* in the case (x > P)
in which the application "is no longer bound by the rules [of the standard]"? 
Doing so places the standard in the very strange position of making a specific
statement about a situation which, by your explanation above, is outside its
scope to make any statement about! :)

I don't get it.

On the other hand, the interpretation earlier suggested -- "cannot use" ==
"must reject" -- offers a straightforward ho-hum reason for why the writers
may have included that specific sentence: Because they desired to specify
how 'conforming applications' -- not 'implementations' -- should behave
specifically in the face of an input ERE which is undefined. One clean,
straightforward, and parsimonious explanation for the phrase "cannot use"
is that that it means "must reject", rather than "you can do what you want".

But, of course, that's just my own opinion. I may well be wrong. You guys
certainly know this spec far better than I, and my take is based only on work
with other IEEE standards. (In which case, as an aside, having chaired a few
subcommittees therein which often dealt with language pedantry like this,
I would never allow "cannot use" to be interpreted in the way that you have
described.) But, that's just me, and it doesn't take into account what you
guys know about POSIX's historical intent that I certainly don't.

Your example of strlen(NULL) does not seem to add clarity to this topic,
because (afaict) there is no specific sentence in the standard regarding
strlen(NULL) which is analogous to the very specific sentence in 9.5.3 calling
out explicitly that conforming applications "cannot use" x > P. The very fact
that that explicit sentence is present in 9.5.3 suggests to me that it was
placed there for a reason, and that one "reasonable reason" might have been
to explicitly say, "conforming apps must not accept such constructs".

Anyway... I will bring this up with the POSIX folks as you suggested. And
again, I apologize for being a fussbudget about this, but I think we have now
homed in on the basic disagreement, and it seems to me essential to understand
it.  Let's see if we can get some clarity from the POSIX ML on it.

Thanks for listening and responding in any case. I do appreciate your time and
will follow up when I learn more from the POSIX ML. (And btw, for sanity/stress
sake: Perhaps worthwhile to view this entire conversation with a certain degree
of humor which the extreme sport of pedantry deserves :) and not get too
irritated about it.  In the end, it may in fact turn out that I have been
wasting your time, but I assure you I am not *trying* to do so. :) I hope you
will eventually smile at all this in the spirit in which it was intended.)