Re: What is the point of bug-lilypond?

[Jean Abou Samra]

Hi James, Werner, all,

Le 14/10/2021 à 08:43, James a écrit :
> Hello,
>
> My own thoughts on this.
>
> On 14/10/2021 01:31, Jean Abou Samra wrote:
> > Hi,
> >
> > After having opened a few GitLab issues in response to
> > bug reports on bug-lilypond, I find James extraordinarily
> > patient for having done this over the years. However, I don't
> > get the value in this system compared to letting people
> > creating issues on GitLab directly. When we transfer an
> > issue to GitLab, it's usually just pasting the text from the
> > email report.
>
> No it is more than that.
>
> It is parsing said bug report - is the thing you are reporting valid? 
> Is it just something a user needs to ask on a different list (e.g. the 
> user is expecting X and is getting Y because they don't understand 
> something but LP is working correctly), is it actually working as 
> designed where the design is not broken but just a hard problem to 
> solve and so on. My day job involves having to tell end-users over and 
> over that X is actually correct because 'reasons' and, at the same 
> time, trying to understand why users think X is a bug when it is not 
> (perhaps improved documentation is all that is needed). So perhaps I 
> am good at this because I have had 20+ years of having to explain this 
> kind of thing to non-technical (or mostly non-interested) people.
>
> Sometimes emails to the bug take the form of 'I do X is this expected 
> or is this a bug' which again requires a slightly different tack.
>
> I would say that 'most' emails to the bug list do NOT need an issue, 
> they are either replies to emails or pointers to existing Issues that 
> explain bug or technical discussions about why something is not a bug 
> but a limitation of what is possible based on how we code things 
> (which itself engenders more emails).


Yes, absolutely -- and you're doing this very well.
However, when you decide that the report calls
for an issue, do you edit it (change example,
use different terms) or is it good enough to
be pasted as-is? I may be wrong, and I certainly
do not want to diminish your work on bug reports,
but I had the impression that most often
the issue was made just from the email thread.
That's also what I usually do.

To be clear, I am not proposing that we change
anything to the way we respond to bug reports
(discussing whether it's a bug, pointing to
documentation, etc.) which is handled very well
as it is. It just appears to me that the bug-lilypond
mailing list is redundant as the same tasks can be
done better via the issue tracker.


> > Right now, bug reports often take a week to be acknowledged,
>
> Maybe but what's the rush? It's not like we have KPIs or Sprints to 
> complete or other arcane methodologies to acknowledge to release LP. 
> The information is there, it isn't going anywhere, if someone fees 
> THAT strongly about wanting conformation (or usually affirmation) then 
> they will just reply to their own emails and someone will look,


Point taken.


> > and some of them are not acknowledged by us at all. Take
> > https://lists.gnu.org/archive/html/lilypond-user-fr/2021-01/msg00014.html 
> >
> > reported 9 months ago, which I just added to the tracker.
>
> Which proves my point although this is not from the bug list this is  
> a user list. So the report is not being put to the correct list in the 
> first place.


Sorry, that one is just silly me doing too
much at a time and pasting the wrong link.
The correct one is

https://lists.gnu.org/archive/html/bug-lilypond/2021-01/msg00001.html

The wrong link I gave earlier is actually the
lilypond-user-fr thread where, as a user
support person, I advised this person to
report the behaviour as a bug -- not doing it
myself to educate to the good practice of
reporting any flaws you find in LilyPond. He did
so, and, as a contributor, I (or anyone else)
didn't react to the bug report. So I feel
this is kind of pointless...

We're all volunteers and nobody is entitled
to respond to bug reports. So if one slips
through the cracks occasionally, I believe
it's better to have it in the database and
keep it for future developers walking around
there than let it be forgotten about. (I do
walk around a lot in the tracker, including
10-year-old issues, and sometimes they give
nice ideas. \vshape was born like that.)


> I like the separation of lists (user, dev, bug) I don't subcribe to 
> the user list at all in fact (I am not that interested) and I knew 
> that some developers don't subscribe to the bug list for their own 
> good reason.
>
> I am not saying we will catch everyone in a timely manner but we do a 
> pretty good job considering (and this bug was reported on the user list).
>
> > Ignoring a report is my opinion the worst of all outcomes since
> > it not only loses information but discourages people to engage
> > in later reports or further investigation. 
>
> Nothing is lost. Also I am not so sure it does discourage engagement 
> that much, if a bug is that big a deal and we miss it, then I am sure 
> that another person would report it, again we've plenty to do as it is.


I was talking about engagement from the user reporting
the bug.

Also, some days I want to see all bugs about a particular
piece of notation (like parentheses) to understand the
context and flaws of the implementation. So the importance
of the bug in terms of how often it manifests itself it
not necessarily an appropriate metric.


> > The information for
> > maintainers of GNU software agree with me:
> >
> > https://www.gnu.org/prep/maintain/maintain.html#Replying-to-Mail
> >
> >    When you receive bug reports, keep in mind that bug reports are
> >    crucial for your work. If you don’t know about problems, you cannot
> >    fix them. So always thank each person who sends a bug report.
> Which we do but this assumes that everything sent to bug email list is 
> a bug (they aren't).


See the following paragraphs though:

   You don’t have an obligation to give more response than that,
   though. The main purpose of bug reports is to help you contribute
   to the community by improving the next version of the program. Many
   of the people who report bugs don’t realize this—they think that
   the point is for you to help them individually. Some will ask you
   to focus on that instead of on making the program better. If you
   comply with their wishes, you will have been distracted from the
   job of maintaining the program.

   For example, people sometimes report a bug in a vague (and
   therefore useless) way, and when you ask for more information, they
   say, “I just wanted to see if you already knew the solution” (in
   which case the bug report would do nothing to help improve the
   program). When this happens, you should explain to them the real
   purpose of bug reports. (A canned explanation will make this more
   efficient.)



> > In spite of us not advertising this as a way to report problems,
> > a number of users have been creating issues on GitLab themselves,
> > likely because that has become enough of a standard in other places.
> >
> > When replies are made on bug-lilypond, someone has to paste
> > them on the ticket as well, for completeness, which leads to
> > an awkward split where one does not know where the discussion
> > is supposed to happen or whether the other person has read
> > a remark on the other channel.
>
> But that has always been a problem and will always remain a problem 
> (again I refer you to the bug report you found on the user list, the 
> French user list as well, which is even more removed) also how many 
> times have things been sent to the other user lists or dev lists or 
> even direct to Han-wen/Jan et al? Again I think this is not that big a 
> deal.

It's not a big deal I would agree. I just think it's
unnecessarily convoluted.


> > So how about retiring bug-lilypond and directing to GitLab
> > instead? Tickets can be triaged there, closing invalid ones,
> > adding a minimal example if not present, perhaps changing the
> > title. The requirement from the same page of GNU guidelines,
>
> I don't see the difference between triaging GitLab vs triaging Bug 
> apart from the fact that it is very easy to use bug-list interface to 
> track email threads - it's so easy to spot of something on the bug 
> list has had a reply or not. I haven't use GitLab's interface that 
> much to follow conversations (which is what 90% of bug is) but if the 
> rest of the interface is anything to go by, I expect it will be a 
> mish-mash of type styles and clours, coloured labels, icons and other 
> tiresome artefacts that plague the modern web these days.
>
> That said, historically we had a group of non-technical people, like 
> myself, that could just use email or use the simple bug-list interface 
> to keep track and check so-called reported bugs. Alas those people 
> have disappeared over the years so the work is being done by us all 
> now who are more technically minded or don't despair about using these 
> 'GitHosty' interfaces.
>
> I can reply-all to emails, can I do the same thing to whatever this is 
> or do I need to use a browser and log in and click somewhere to open a 
> window to type my reply and then and then and then ...?


The issue tracker on GitLab is pretty much like
the interface for merge requests, which you are
using for the countdown. One can also reply to
notifications by email, and change status with
quick actions in the body of the email. There is
an almost one-to-one mapping between reactions
by email and on GitLab:


=====================      =========================
Email on bug-lilypond      Issues on GitLab
=====================      =========================

[Open issue and add        Thank you for your report.
labels]                    /label ~"Defect" /label ~"Regression"
Thank you for your
report, I have opened
issue #abcd.


Thanks for your message,   Thanks, but this duplicates
this is already known      the existing issue #abcd.
as issue #abcd.            /duplicate #abcd

                           [Note that GitLab displays similar
                           tickets when opening one, which can
                           help reducing the number of duplicates.]


Paste comment received     Nothing to do.
from the list on the
issue.


Thanks, but this is not    Thanks, but this is not
actually a bug, please     actually a bug, please read
read the documentation     the documentation at ...
at ...                     /close /label ~"Invalid"


Please ask support         Please ask support questions
questions  on              on the lilypond-user list instead
lilypond-user rather       of creating issues.
than bug-lilypond.         /close /label ~"Invalid"


Example interaction: https://gitlab.com/lilypond/lilypond/-/issues/6157


> There are two point of views.  Some people (for example, the HarfBuzz
> developers) don't want e-mail on their mailing list; they prefer that
> users open issues for both user questions and bug reports.  I guess
> this works because HarfBuzz is a very specialized library, not to be
> used by Joe User but by developers who already know how good bug
> reports should look like.
>
> The other point of view is that bug reports should be collected in a
> database, being correctly tagged and as concise and compact as
> possible.  I agree with James that it makes sense to parse and filter
> reports in advance to achieve that, so I'm on his side.


I'm not proposing to direct support questions to
GitLab issues. Rather, I have the impression that
bug-lilypond is redundant with the other channels we
have.

- If you have a question, ask on lilypond-user.

- If you experience strange behavior and you do
  not know whether it's a bug (often you will also
  be looking for a workaround), post on lilypond-user
  similarly.

- If you have general thoughts about the way
  X could work better or want design feedback,
  post on lilypond-devel.

- If you have a precise bug with minimal example,
  or a well-defined enhancement request, open
  an issue.

A factor to consider is that, as far as I can read
http://lilypond.org/bug-reports.html (but I can't
remember how it worked for myself), posting to
bug-lilypond requires being subscribed to the list,
thus receiving all subsequent bug reports, which is
not what a user wants. On GitLab, just turn on
notifications for an issue if you are interested
in it. This is turned on by default if you are
the author of the report.

It's also a question of communication. With bug-lilypond,
a bug report is supposedd not to immediately interface with
developers (for some definition of this term). A drawback
of GitLab issues is that everyone there receives notification
for all reports including those that are not actually
valid. I would actually turn this to an advantage:
it's often interesting to see common failure patterns
(leading to better interfaces and/or better documentation).

How about experimenting with it? We can recommend
GitLab for bug reports on the website during two months
and see how it fares.

Best,
Jean