Re: issue verification

[Jean Abou Samra]

Hi,

Le 17/09/2020 à 13:27, Michael Käppler a écrit :
> Am 17.09.2020 um 09:47 schrieb Jonas Hahnfeld:
>
> > Hi all,
> >
> > unless I'm missing something, there has been no activity on this since
> > the community effort in May [1][2] and issues since version 2.21.2
> > don't yet have Status::Verified labels. As I'm currently updating CG to
> > mention our use of milestones (not sure why I forgot about this
> > previously), the verification procedure should probably also get
> > updated.
> > I haven't been active in this area (and don't plan to be), so it would
> > be great if somebody could look over the current description (assuming
> > the procedure should be continued).
> >
> > To give a head start, here's again the list of all Status::Fixed:
> > https://gitlab.com/lilypond/lilypond/-/issues?scope=all&utf8=%E2%9C%93&state=closed&label_name[]=Status%3A%3AFixed 
> >
> >
> > Furthermore you can directly get to the issues associated with a
> > milestone, for example for 2.21.2:
> > https://gitlab.com/lilypond/lilypond/-/milestones/320#tab-labels
> >
> > Thanks,
> > Jonas
> >
> > 1: 
> > https://lists.gnu.org/archive/html/lilypond-devel/2020-05/msg00284.html
> > 2: 
> > https://lists.gnu.org/archive/html/lilypond-devel/2020-05/msg00379.html
> Hi Jonas,
> I just took a quick glance on it and have many questions... :)
> Let me summarize them with an example:
>
> Take
> https://gitlab.com/lilypond/lilypond/-/issues/5999
>
> 1. Federico wrote in the thread you mentioned [1]:
> "In the last comment you should find a commit id (if it's missing you'll
> have to search it)."
>
> I think this refers to the time before Gitlab, when issues had
> a message like
>
> Trim unused toplevel targets.
> author    Han-Wen Nienhuys <hanwen@lilypond.org>
>     Sun, 22 Mar 2020 00:16:36 +0100 (00:16 +0100)
> committer    Han-Wen Nienhuys <hanwen@lilypond.org>
>     Tue, 31 Mar 2020 21:18:36 +0100 (22:18 +0200)
> commit    0755e2f3dbb99ac256fb447d6ed14d3fe87b8ab5
>
> at the end of the thread, right?

I think so.

> 2. So I thought I had to look at the corresponding MRs to find
> the commits I shall verify.
> For #5999, there are two MRs listed as 'Related', which are:
>
> https://gitlab.com/lilypond/lilypond/-/merge_requests/145
> Add PDF bookmark support & multi-level TOC outline ability
>
> and
>
> https://gitlab.com/lilypond/lilypond/-/merge_requests/147
> Avoid scm_append_x for property values
>
> I'm not sure how !145 is related to #5999. Maybe #5999 did show up
> after !145, and !147 did fix it.
>
> Anyway, am I right that I have to extract all commit ids from both MRs
> and check if they got into 2.21.2? Or do I have some faulty thinking 
> here?
> This would be very complicated and time-consuming, especially when MRs
> consist of several commits.
>
> 3. What I do not get: A milestone is simply a named period of time, to 
> which
> you can assign MRs and issues, right?


As far as I understand, that's it, roughly. Milestones are just a kind 
of extended
label concept (I think they integrate with the release feature somehow, 
too). At any
rate, the documentation is over there:
https://docs.gitlab.com/ee/user/project/milestones/

> So after each release, you (or
> someone else)
> start a new milestone with the next version number. If an issue has been
> closed, you
> set the issue's milestone to the current milestone.
> An issue is closed automatically when the corresponding MR has "Closes
> #..." in the description.
> So if an issue is closed and the milestone of the next release is added,
> what is there left
> to verify? Can't we be sure that the commits of the corresponding,
> closed MR have gone in?
>
> Michael

In my opinion, we no longer need to verify patches. That is, there is
no need to check that the commits are present in master using committishes.
This was done when branches were manually managed, meaning that a patch 
could
have slipped through because of a wrong usage of Git; I hardly see how
something could go wrong on GitLab.

Basically, the new setup for me would mean installing the development
release (downloaded from lilypond.org, not self-compiled, because
build issues are a non-negligible source for problems -- the Contributor's
Guide prescribes this), and going through all Status::Fixed issues,
testing wether the bug is effectively gone. And if there is a need for a
regression test or more documentation, opening a new issue for this.

What I do not yet understand is when milestones should be assigned
to issues and MRs. So far Jonas has done this from time to time. That was
after each release, right? Should we switch to doing this as part of the
verification process?

By the way, verifying regression tests is also in order…

Thoughts? I can help a bit next week-end.

For reference, here's the relevant CG chapter, which is to be updated:

http://lilypond.org/doc/v2.21/Documentation/contributor/bug-squad-checklists

Thanks,
Jean