bug#20693: SKIP not counted in TAP

[Arthur Schwarz]

Thanks Nick;

But doesn't this mean that tap-driver.sh didn't report a violation to the
TAP standard? And why is my output intermixed in the .trs file? If 'skip'
below is illegal, why isn't it tagged? (I think that multiline comments need
leading whitespace?)

As another issue, I looked at the awk code and have some potential revisions
below. I know very little about awk but I think the resultant code is faster
in that the 'is_first_read = 1' is not necessary and consequent checks for
each line input are unnecessary. 

As to your (sly and underhanded) comment about awk being defined (and I
should read it), well, in this case I agree. But in general where something
is important to the issue at hand obligating the user to read every relevant
manual bearing on the issue I feel is burdensome and unnecessary. I
understand the problems with following 'other peoples manuals' and burden
that that places on the maintenance staff, but requiring that readers of the
Automake User's Manual read the Autoconf, Automake, make, TAP standards,
bash for 'env' and etc. is needless. My opinion is that some of the material
in these (and other manuals) which have a direct bearing on Automake should
have more than a reference to Yet Another Manual. This issue directly bears
on, for example, TAP, where a few wise words from the Master's Of The
Universe will go a long way being productive. But, this is an opinion. Oh, I
am doing some writing for Section 15 of the Automake Manual to say that I am
putting some effort is providing an alternative viewpoint.

===================== tap-driver.sh =====================

   
   ## --------- ##
   ##  PARSING  ##
   ## --------- ##
   
   st = getline
   
   nextline = $0;  # I have no idea what $0 does.
   
   if (st < 0) # I/O error.
     abort("in input loop: only one input line")
   }
   
   do
     {
       # Involutions required so that we are able to read the exit status
       # from the last input line.
       # Copy any input line verbatim into the log file.
   
       o o o
       
       st = getline
       curline = nextline
       nextline = $0
       $0 = curline

   while(st >= 0)

   if (st < 0) # I/O error.
      fatal("I/O error while reading from input stream")
   }
   
On 2015-05-29 08:15 -0700, Arthur Schwarz wrote:
> skip causes the following issues:
> 1: It is not reported.
> 2: It is not summarized.
> 3: It must be subtracted from the plan (tests run), and
> 4: The .log file has TAP generated output and the test data intermixed.
> 
> I think this is against the TAP definition?

In a way, because...

> skip
> ok
> PASS: test2.tap 1
> not ok
> FAIL: test2.tap 2
> 1..2

... it is your program's output that does not follow the specification.

There are two basic styles of skipping in TAP.  You can skip a single
test result:

  ok 1 # skip reason

or you can use the plan line to indicate that the entire test program is
to be skipped:

  1..0 # skip reason

Regards,
-- 
Nick Bowler, Elliptic Technologies (http://www.elliptictech.com/)