Re: regtest not catched by make check

[Carl Sorensen]

We used to have a pdf compare.   I guess that took too much time so we dropped 
it.  Maybe we could reinstate it for a prerelease check that is run manually?

Sent via the Samsung Galaxy S®6 active, an AT&T 4G LTE smartphone
Get Outlook for Android<https://aka.ms/ghei36>

________________________________
From: lilypond-devel 
<lilypond-devel-bounces+carl.d.sorensen+digest=gmail.com@gnu.org> on behalf of 
Jonas Hahnfeld via Discussions on LilyPond development <lilypond-devel@gnu.org>
Sent: Saturday, November 7, 2020, 11:30 AM
To: Thomas Morley; Michael Käppler
Cc: lilypond-devel
Subject: Re: regtest not catched by make check

Am Samstag, den 07.11.2020, 17:30 +0100 schrieb Thomas Morley:
> Am Sa., 7. Nov. 2020 um 15:35 Uhr schrieb Michael Käppler <xmichael-k@web.de>:
> >
> > Am 07.11.2020 um 13:23 schrieb Thomas Morley:
> > > Hi,
> > >
> > > it would be nice to have a regression test for
> > > https://gitlab.com/lilypond/lilypond/-/merge_requests/497
> > >
> > > I think I found some way to display differently with and without the
> > > fix. See the attached file.
> > >
> > > Though, I can't convince 'make check' to catch the difference.
> > Hi Harm,
> > could you please try to run
> > `lilypond -ddump-signatures' on your regtest, with and without the fix
> > and compare
> > the .signature files?
> >
> > Cheers,
> > Michael
> > >
> > > Currently I'm at a loss. Any hints?
> > >
> > > Thanks,
> > >    Harm
> >
>
> Hi Michael,
>
> I followed your advice and applied a diff between the two .signature-files.
> With no output. They are equal...
>
> Though, the pdf's aren't. See attached.
>
> Obviously I don't understand what 'make chack' is looking for...

As far as I understand `write-system-signature' in scm/stencil.scm, the
signature files have the positions of all grobs and their extents
*before* applying stencils.
What about just having a simple test that would fail visual inspection?
When looking through the list of all regression tests, it should be
pretty catchable if the description says that all objects have a
certain color, but they don't.

Jonas