Re: [lmi] Relaxing BOOST_TEST_THROW() message test to do substring match only

[Vadim Zeitlin]

On Thu, 18 Feb 2016 02:26:39 +0000 Greg Chicares <address@hidden> wrote:

GC> On 2016-02-18 01:37, Vadim Zeitlin wrote:
GC> 
GC> >  IMO checking for substring, instead of exact, match, would be a better
GC> > solution because it wouldn't really weaken the existing tests and would
GC> > make using "" much less appealing thus strengthening the future ones. And
GC> > as a nice side effect, this would also make the macro implementation
GC> > simpler as it wouldn't need to worry about "\n[file" part.
GC> > 
GC> >  What do you think?
GC> 
GC> Could we make this optional?

 Yes, of course. I can see 2 reasonable ways to do it: the simplest one is
to just add another macro working as I'd like it to, and the most difficult
problem with this approach is choosing the name of this macro: we could use
a direct BOOST_TEST_THROW_SUBSTRING or more descriptive (but less precise)
BOOST_TEST_THROW_RELAXED or something in the middle but very long like
BOOST_TEST_THROW_INCLUDING_MESSAGE.

 Another one would be to make the third argument of the existing macro an
object of some exception_message_matcher type. Such object would be
implicitly constructible from string and would match exactly in this case,
for compatibility, but it could also be constructed using some
match_substring() factory function which would create an object matching as
I'd like it to.

 The second approach is slightly more complicated but more flexible (e.g.
we could easily extend it to support regex matching in the future) and
avoids the need to agonize over the macro naming decision, so I prefer it.
The main problem with it from your point of view is probably that it's
intrusive, i.e. it requires changes to the existing code instead of being a
pure addition as the first one.

GC> Then I can have the rigid tests I like, and you can have greater
GC> flexibility. And who knows, I might someday come round to your point of
GC> view.

 Just as an example of why I think it's an overkill to test for the exact
match: if I want to check that the SOA table creation code doesn't accept
tables with the wrong hash, it's quite enough to check that an exception
with "hash value" somewhere in its message is thrown, it really doesn't
matter if the message says "hash value is incorrect" or "hash value
mismatch" or "expected hash value XXX but got hash value YYY" which could
all be reasonable formulations for this exception. And it definitely
doesn't matter if the full message is actually "Invalid data for table
NNN: incorrect hash value XXX specified at the line LLL", the fact that it
includes "hash value" is quite enough.

GC> Now that we have <regex> in libstdc++, would you want to test regexen
GC> instead of substring matches? E.g.: /failure on line [0-9]+/ if the
GC> exact line is irrelevant or somehow indeterminate?

 We could do this but IMHO it's unnecessary right now. With the second
approach above it would be simpler to do it in the future if we change our
mind.

 Please let me know which approach would you prefer,
VZ