Re: Problem running functional tests from China

[Michal Suchánek]

On Wed, Feb 24, 2021 at 12:57:13PM +0000, Daniel P. Berrangé wrote:
> On Wed, Feb 24, 2021 at 01:48:25PM +0100, Philippe Mathieu-Daudé wrote:
> > On 2/24/21 1:08 PM, Daniel P. Berrangé wrote:
> > > On Wed, Feb 24, 2021 at 01:01:40PM +0100, Michal Suchánek wrote:
> > >> On Wed, Feb 24, 2021 at 12:37:53PM +0100, Philippe Mathieu-Daudé wrote:
> > >>> On 2/24/21 12:06 PM, Michal Suchánek wrote:
> > >>>> On Wed, Feb 24, 2021 at 10:29:15AM +0100, Philippe Mathieu-Daudé wrote:
> > >>>>> Just to inform the files hosted on github don't work when
> > >>>>> testing from China:
> > >>>>
> > >>>>>
> > >>>>> raw.githubusercontent.com resolves to 127.0.0.1
> > >>>
> > >>> Actually not even localhost but 0.0.0.0 :)
> > >>>
> > >>>> And from where does this come from?
> > >>>>
> > >>>> Your local system, your ISP, ... ?
> > >>>
> > >>> Probably the ISP:
> > >>>
> > >> ...
> > >>>
> > >>> I raised that problem not to find a way to bypass an ISP
> > >>> firewall, but to see if there is a way to use another
> > >>> storage for test artifacts so all the community can run
> > >>> the tests.
> > >>
> > >> I don't think protecting from random network malfunction is something
> > >> the qemu project can do.
> > >>
> > >> That said, downloading the test data during test run does indeed look
> > >> fragile.
> > >>
> > >> We have the concept of git submodules which is used in qemu extensively
> > >> so the test data could possibly be included directly in the git tree.
> > >>
> > >> The obvious downside is that the current and past test binaries will take
> > >> up disk space for all users that check out the repository, even those
> > >> that don't run the tests.
> > > 
> > > We explicitly do not want to be distributing / including the
> > > images ourselves, as that makes the QEMU project responsible for
> > > license compliance and thus provision of full & corresponding source.
> > 
> > I am not asking that QEMU redistribute these files, I'm asking to the
> > community if someone know an alternative to store files so we can run
> > QEMU tests in Chinese cloud farms.
> > 
> > Back to your comment, should we remove these tests?
> 
> No, I don't think that we should do that.
> 
> If someone has broken ISP and they need to run all tests, then they'll
> need to use a VPN.

I would say running tests offline is generally desirable feature - runs
in secure environments, networking errors not affecting test results,
etc.

Also pulling test data from a number of random repositories does not
sound great for test reproducibility. Is there some plan for addressing
these random artifacts going away some day other than ignoring the tests
for which test data is missing?

Thanks

Michal