Re: [PATCH 2/2] testsuite,pickles: Re-write test for pickles using `unittest.pk`

[Jose E. Marchesi]

>> 2) What about XFAIL, XPASS and the like.  Right now the test harness
>>    catches E_assert and assumes it is a fail, but we have to cover the
>>    other possibilities too.  We need to find a way to convey that
>>    information to the test harness.  I have a change for the Exception
>>    struct in mind, to put the location information in its own field
>>    instead of `msg'.  But for now we are ok with just FAIL and PASS.
>> 
>
> In `unittest.pk` (in next patch `pktest.pk`) pickle, every exception is
> considered as failure (not only assert ones).
> User should take care of him/her expected exceptions.

Yes, that makes sense.  There is not a catch-them-all handler in
tests_run.

> And I don't think we need XFAIL, XPASS.
> Expected failure can be expressed using negating the condition:
>
>     assert (do_fail () != OK, /*error message*/ "unexpected success");
>
> Or if exceptions are involved:
>
>     var A = [1,2,3];
>     try
>       {
>         A[3];
>       assert (0, "unreachable reached!");
>       }
>     catch if E_out_of_bounds
>       {
>         /* assert (1, "expected exception"); */
>       }
>
> or,
>
>     try
>       {
>         raise_exception ();
>       assert (0, "unreachable reached!");
>       }
>     catch (Exception ex)
>       {
>         if (ex == E_out_of_bounds || e == E_io)
>         assert (1, "expected exception");
>       else
>         assert (0, "unexpected exception: " + exception_str (ex));
>       }
>
>
> I think, this approach is simple and general.
> This also can be documented in the manual to guide users how to write tests.

Yes, but look at the TAP protocol I pointed you to yesterday.  It
supports xfail, xpass, unresolved, etc.  We can work with just pass/fail
for now, but eventually we will need a mechanism to convey other status
in the asserts, like "unresolved".