Re: Treating tests as special case

[Pjotr Prins]

On Thu, Apr 05, 2018 at 08:05:39AM +0200, Gábor Boskovits wrote:
>    Actually running tests test the behaviour of a software. Unfortunately
>    reproducible build does not guarantee reproducible behaviour.
>    Furthermore there are still cases, where the environment is
>    not the same around these running software, like hardware or
>    kernel configuration settings leaking into the environment.
>    These can be spotted by running tests. Nondeterministic
>    failures can also be spotted more easily. There are a lot of
>    packages where pulling tests can be done, I guess, but probably not
>    for all of them. WDYT?

Hi Gabor,

If that were a real problem we should not be providing substitutes -
same problem. With substitutes we also provide software with tests
that have been run once (at least).

We should not forbid people to run tests. But I don't think it should
be the default once tests have been run in a configuation.

Think of it as functional programming. In my opinion rerunning tests
can be cached.

My point is that we should not overestimate/overdo the idea of
leakage. Save the planet. We have responsibility.

Pj.