Re: Status of the test framework project

[Greg Williamson]

Running continuous integration on Savannah isn't really an option. If the project moves to gitlab as was discussed months ago. I would not be opposed to porting my azure configs to gitlab ones. However it is not really practical for me to set up servers for your organization. The scripts I wrote should run on any Linux machine though. "Improving the UI" is really vague so it would be helpful to me or your future student if you're clearer about what you want upfront. I don't know much about bcmp or whatever but switching out the comparison tool used in the scripts shouldn't be difficult.

On Sun, Jan 10, 2021, 9:44 AM Werner LEMBERG <wl@gnu.org> wrote:

> What is the status of the test framework project?

Dormant.

> It was attempted during GSoC 2020 but the submitted code does not
> satisfy one of the design criteria: just work.  Several parts of the
> project can be improved.

I fully agree.  Collaboration with the student was difficult,
unfortunately.

> - Expensive Microsoft Azure nodes can be replaced with local
>   GNU/Linux boxes. While the computing requirements of the project
>   are moderate, keeping raster data in cloud storage is
>   unaffordable.
>
> - The general purpose image comparison can be replaced with more
>   finely tuned solutions. The bmpcmp utility that is used by
>   Ghostscript developers is a good starting point.
>
> - The UI can be more sophisticated.
>
> - There is no test case collection.
>
> - The project is abandoned by the author. There's no new commits
>   after the official end of the project.

I fully agree with your analysis.

> If the test framework project is re-offered in 2021, it could be
> done better.

Yes, probably.  This would be the third attempt, then :-)

AFAICS, what was done during GSoC 2020 looks promising, so we would
need a dedicated student to continue the work.

By the way, GSoC 2021 will work differently: The available time for
students gets halved; this implies smaller projects.  In other words,
what you suggest might indeed be a good solution.


    Werner