Re: master 262d0c6: Mark some tests as expensive

[Michael Albinus]

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>> Having people do "make check" as a matter of routine, and running 97% of
>> the tests is, in my opinion, better than people doing a "make check"
>> seldom, but running 98% of the tests.
>>
>> Where the cutoff is a matter of balance, yes, but I think a test that
>> takes a second is way too slow to be run in the routine case.
>
> FWIW, I think The Right Solution is to label each test with its "cost"
> (which could be auto-computed by a script that runs those tests to
> measure that cost).  Then we can provide a command to run all tests
> cheaper than X, and the end-user gets to decide where is "the cutoff" in
> his balance.

Nice idea. The problem will be to decide what are the default costs we
accept running "make check". I fear that there will be voices to make
the default costs such low, that (for example) auto-revert tests are
kicked off. As we have now.

>         Stefan

Best regards, Michael.