Re: [lmi] 'authenticity_test' broken years ago?

[Vadim Zeitlin]

On Thu, 26 Mar 2020 23:00:57 +0000 Greg Chicares <address@hidden> wrote:

GC> Vadim--Can you explain this puzzle? With or without
GC>   https://github.com/vadz/lmi/pull/117
GC> one particular test in 'authenticity_test' misbehaves,
GC> though in a way that (defectively, I suppose) isn't
GC> counted as an error.

 The short answer is that AFAICS this test has been intentionally written
like this and it doesn't misbehave at all.

GC> It occurs here:
GC> 
GC>   /// Postcondition: the file passes a test with the 'lmi_md5sum' program.
GC> 
GC>   void PasskeyTest::InitializeMd5sumFile() const
GC>   {
GC> ...
GC>       std::string s = "lmi_md5sum --check --status " + 
std::string(md5sum_file());
GC>       system_command(s);

 It doesn't occur here however. The check done here succeeds, as expected.

GC> and I would expect that command to succeed, printing something like
GC>   coleridge: OK

 It doesn't print anything because of the presence of the "--status"
option, which suppresses normal output.

GC> but here's what I observe, in all cases running wine+debian-testing;
GC> first, without PR117:
GC> 
GC> ---->--BEGIN-->----
GC> /opt/lmi/src/lmi[0]$make unit_tests unit_test_targets=authenticity_test.exe
GC> make[2]: Nothing to be done for 'build_unit_tests'.
GC> List of unit-test targets that did not build successfully:
GC> 
GC> List ends.
GC> 
GC> Running authenticity_test:
GC>   Result of 'md5sum --version':
GC> md5sum (PCP patchlevel 2) (GNU textutils) 1.22
GC> Exit code 1 from command 'md5sum --check --status validated.md5'.
GC> 
GC> ... 76 tests succeeded
GC> ----<--END--<----

 The "Exit code ..." line in the output comes from Authenticity::Assay()
call in PasskeyTest::TestDataFile(), which is expected to fail. It would
probably be useful to either suppress the error output or to prepend it
with the same "Ignore the error in the next line, it is expected" message
as is already used in other places. I could, of course, make a patch for
either of those alternatives if desired.

GC> How can this be?

 I hope the above answers this question, but please let me know if it
doesn't.

 FWIW I did have similar problems, when I thought something happened in one
test when it actually happened in a completely different one, with lmi
tests in the past. Not being able to run just a single test doesn't really
help. As I don't think there is any chance of starting to use a third party
tests runner in lmi (such as my current favourite CATCH), it might be at
least worthwhile considering extending the existing mini-framework to allow
specifying the tests to run on the command line. IME it's a really useful
feature to have and if you could focus on a single test in this particular
case, you would have discovered the problem immediately.

 Regards,
VZ

pgp309090WJkW.pgp
Description: PGP signature