bug#24402: should-error doesn't catch all errors

[Alex]

npostavs@users.sourceforge.net writes:

> Alex <agrambot@gmail.com> writes:
>
>> npostavs@users.sourceforge.net writes:
>>
>>> Yes, ert binds `debugger' in order to get full backtrace information
>>> when there is an error.  This means it won't see errors caught by
>>> condition-case.  That's good when it ignores errors caught by test code
>>> using condition-case, but does give rise to problems.  There is some
>>> relevant discussion in Bugs #11218 and #24617.
>>>
>>> Espcially the suggestion in #24617 of using `signal-hook-function' to
>>> record error info instead of using `debugger', I think doing this could
>>> simplify things a lot.  It is definitely going to require messing around
>>> with ert's internals though...
>>
>> Thanks for the info. I may have discovered a workaround, but I'm not
>> sure if there's any negative side-effects. All the tests pass, though.
>>
>> What do you think of it? It's obviously not ideal, but I think it at
>> least fixes the issues at hand.
>
> Does it also work when loading the elc version of the test file?  (try
> 'make check TEST_LOAD_EL=no')

Oh, it doesn't load the elc version by default? That's surprising; I
think that should be documented in the test README.

I get 3 test failures with TEST_LOAD_EL=no, but I don't believe they're
because of me. On a mostly clean master (d014a5e15) those 3 also error.
One of them is simple to fix (the (require 'subr-x) should not be inside
eval-when-compile in dom-tests). The other failing tests are
subr-test-backtrace-integration-test and cl-lib-defstruct-record.

> What about tests like this?
>
>     (ert-deftest check-error-handling ()
>       (should
>        (eq 42
>            (condition-case ()
>                (/ 1 0)
>              (arith-error 42)))))

It works for me, yes. As long as `debugger' is set to a symbol. I can
make it a bit more robust by using an additional defvar in
ert--run-test-internal.

Are you asking because it doesn't work for you?