Re: branching stable/2.22?

[Jonas Hahnfeld]

Am Montag, den 24.08.2020, 12:56 +0200 schrieb Jean Abou Samra:
> Hi,
> 
> > Le 24 août 2020 à 08:30, Jonas Hahnfeld <hahnjo@hahnjo.de> a écrit :
> > 
> > Am Sonntag, den 23.08.2020, 23:44 +0200 schrieb Jean Abou Samra:
> > > Hi,
> > > 
> > > (Sorry about the strange reply style.)
> > > 
> > > > On Sun, Aug 23, 2020 at 1:58 PM Jonas Hahnfeld <hah...@hahnjo.de> wrote:
> > > > I'd like to ask what it would take in principle to branch stable/2.22
> > > > and what others think about this.
> > > > 
> > > > Personally I'm convinced that creating the branch and only picking bug
> > > > fixes from master is the only guaranteed way to stabilize. Now you
> > > > might say that there were only few unstable releases (AFAICT there was
> > > > 2.19.65 before branching stable/2.20). However, I think there are
> > > > already quite some user-facing changes and also the switch to Python 3.
> > > > With Python 2.x being EOL since January, it's only a matter of time
> > > > until Linux distributions and BSDs want to drop that, and it
> > > > would be
> > > > unfortunate if that put a (temporary) end to providing LilyPond for
> > > > their users. If we had release candidates or even a stable version
> > > > until then, it would definitely help.
> > > 
> > > Maybe we could try to release 2.20.1 with Python 3?
> > 
> > That would mean porting 50+ commits which sounds like a bad idea and
> > even gets worse because of the reformatting in master. The latter
> > implies that any bug fix made in master will result in merge conflicts.
> > Plus I don't understand the proposal if you're at the same time saying
> > that the scripts are fragile. By that logic, why would we backport such
> > extensive changes and claim they're stable?
> 
> Right, I was oblique: the scripts are fragile at present, so branching 
> release/2.22 now is no good in my opinion, but hopefully we can stabilize 
> them faster than we stabilize LilyPond as a whole, and have that in 2.20.1 or 
> 2.20.2.
> 
> Can you explain why porting 50 commits from master to 2.20 is a bad idea?

I think it's a bad idea because it goes against my basic understanding
that only bug fixes should be ported a stable branch. Here's the total
number of commits in stable/2.20 since branching:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 | wc -l
588

Sounds high at first, but most of that was translations, ie
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- $(ls | grep -vE 
"Documentation|po") | wc -l
268
If only focusing on actual code changes:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- flower/ lily/ ly/ 
python/ scm/ scripts/ | wc -l
198
with the majority in "core" LilyPond:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- lily/ ly/ scm/ | wc 
-l
148

> If we port all Python related-commits (including the reformatting), there 
> won't be any merge conflicts, or am I being dense somehow?

Consider:
$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- python/ | wc -l
22
vs
$ git log --oneline HEAD...release/2.19.65-1 -- python/ | wc -l
115

and 

$ git log --oneline release/2.20.0-1...release/2.19.65-1 -- scripts/*.py | wc -l
5
vs
$ git log --oneline HEAD...release/2.19.65-1 -- scripts/*.py | wc -l
67

Being the individual with the most commits in there during the cycle, I
strongly advise against taking all of this for a minor stable release.


> I do understand that having LilyPond 2.20.0 support exclusively Python 2 but 
> 2.20.1 be Python 3 only feels weird. However, I value the interest of the 
> average user more than that of packagers. Neither Python 3 nor Guile 2 is a 
> breaking change from the user's point of view. If having LilyPond 2.20.1 (or 
> 2.20.2) support Python 3 is not feasible, I think we should just let 
> distributions drop LilyPond (see below). I'm not happy about it, but this is, 
> in my opinion, preferable to making a stable release, in a hurry, that will 
> contain more bugs and few user-facing changes.
> 
> > > > The same can of course happen with Guile, but that situation has been
> > > > going on for years. Furthermore, it's at least possible to compile and
> > > > use current master with Guile 2.x, even if slower. In my understanding,
> > > > things are getting better but properly integrating byte compilation is
> > > > still a good amount of work that nobody could give a deadline on.
> > > > 
> > > > WDYT?
> > > 
> > > [Han-Wen] I think that both Python 3 support and usable (if imperfect) 
> > > GUILE 2 support is a strong argument for releasing a new stable version 
> > > soon.
> > > 
> > > Why Guile 2? If it's still imperfect and slower, we don't want to make it 
> > > the default in the binaries at lilypond.org, do we? So how will the 
> > > situation be different from 2.20? Sorry, I must be missing something 
> > > obvious here... I don't understand you.
> > 
> > At least in my book, it's not about changing the default but at least
> > making it possible for distributions to compile with Guile 2.x instead
> > of just throwing LilyPond out.
> 
> Does this mean that we'll receive bug reports that won't be reproducible by 
> others because they'll actually be related to Guile 2? In my opinion, the 
> distributions just throwing out LilyPond is better.
> 
> Additionally, the sh installers are recommended by the official website over 
> distribution package managers.
> 
> > > At any rate, Python 3 support is great, but the scripts are fragile at 
> > > the moment. This is clear from the tracker and the bug-lilypond list. Our 
> > > Python scripts still need fixes in the way we distribute them, plus 
> > > encoding-related issues (I'm planning on tackling the latter point in a 
> > > short period at the end of August, but who knows what that will reveal).
> > 
> > Could you please point to concrete issues? The distribution of scripts
> > hasn't changed, so it basically still works the same it did for years
> > (decades?).
> 
> I'm writing this with a browser that is too old to view gitlab.com. There is 
> at least
> https://www.mail-archive.com/bug-lilypond@gnu.org/msg43376.html
> 
> which looks serious at first sight.
> 
> > > [Han-Wen] I've been working on the build system (obviously), with in the 
> > > back of my mind having a build that is no longer recursive, but that work 
> > > could be paused for a bit while we prepare for releasing a 2.22. Is there 
> > > a list of problems in the current master that have to be resolved?
> > > 
> > > Problems are basically popping every week (e.g., info installation, 
> > > translation tools, etc.). You're fixing them every week, which is really 
> > > great, but before creating a release branch that is devoted to stability, 
> > > I think we need a couple months to see what new problems appear, don't we?
> > > 
> > > We have unstable releases to publish new features and get testing. In my 
> > > opinion, stable releases should really focus on stability, there's no 
> > > need to rush because of Python 3 and Guile 2.
> > 
> > I don't think we'll get testing from distributions until we declare a
> > way to stabilize.
> 
> We're speaking from different points of view here: in my book, our main 
> source of testing is our development and beta releases brought to users 
> through installers. I mean that LilyPond 2.22 should introduce full support 
> for Guile 2 with byte compilation, probably dropping Guile 1 support too, and 
> we get Guile 2 testing from those who try out the betas.

I seriously doubt you'll get that for 2.22 next year.

> > > At least four areas are currently under flux: Python scripts, the build 
> > > system, Guile 2 support, and fonts (Owen's project), and I don't see that 
> > > master is coming any close to stability. I think we are better with 
> > > focusing on these areas as long as they still require substantial 
> > > attention, so as to get a stable and nice LilyPond 2.22 in a timely 
> > > manner. You derecursify the build, David completes his stack of rather 
> > > extensive purportive changes, Owen merges the GSoC branch, etc., and only 
> > > then it'll be time to care about LilyPond 2.22.
> > 
> > And that's my point: If we just sit and wait, the next stable release
> > might happen after all major distributions threw LilyPond out - simply
> > because it cannot be built in a world of Python 3.
> > > Am Sonntag, den 23.08.2020, 14:59 +0200 schrieb David Kastrup:
> > > > > I'd like to ask what it would take in principle to branch stable/2.22
> > > > > and what others think about this.
> > > > 
> > > > I don't see that this is a good point of time.
> > > > There has been an influx of badly tested changes to the build system and
> > > > directory setup and the web pages that has to stabilise and result in a
> > > > workable version of LilyPond. I don't see the point in branching a
> > > > "stable" branch if there is so much in a destabilised state: you'd have
> > > > to cherry-pick loads of stuff from the unstable branch as it comes in.
> > > 
> > > [Jonas] I fully agree 
> > > 
> > > ... and so do I (for what my opinion's worth, really) ...
> > > 
> > > [Jonas] and I should have been more clear that I don't expect the branch 
> > > to happen in the next week. The point was to find out what it would take 
> > > because just waiting for some unspoken condition to become true is not 
> > > exactly going to happen without some effort.
> > > 
> > > What about scheduling the release?
> > > 
> > > While I do know that "Grass doesn't grow faster when you pull on it.", I 
> > > would definitely like having a defined point in time where the stable 
> > > release is to happen, so that everyone can focus on bug fixes before it 
> > > happens. Sure, we aren't going to get agreement in a second about the 
> > > date (even if not more precise than a month), but to me, having this talk 
> > > now is preferable so as to give LilyPond development a tempo. To say it 
> > > with other words, we've got a score to play; arguing about the tempo is 
> > > better than starting the piece with different tempi.
> > > 
> > > As sort of a shot in the dark, how about planning the 2.22 release for 
> > > May 2021, for example?
> > 
> > Do you mean branching stable/2.22 or releasing 2.22.0 in May 2021? In
> > my understanding, the past process includes the release of beta
> > versions from the branch. That makes it close to impossible to predict
> > the final date of the stable version, and that's not the purpose of
> > this thread.
> 
> I mean releasing 2.22.0 in May 2021. This is not about predictions, but 
> objectives. I think that instead of planning each small step on the fly with 
> no idea how the future looks like, we should settle an expected date for the 
> release and plan backwards accordingly. Sure, there could be critical bugs 
> that delay the actual release, but setting the expectation enables more 
> resources to focus on the release by the time it is due to happen. In my 
> opinion, this is the way we can avoid things like the 2.14 release that is 
> documented in the CG.
> 
> So, an expected release in May 2021 would mean branching release/2.20 around 
> January (?) and beta releases at a monthly cadence until the release is out.
> 
> I'm curious about what others think of that. In fact it looks like you 
> already proposed something along these lines:
> https://www.mail-archive.com/lilypond-devel@gnu.org/msg72997.html

And it didn't get much support. Which is why I don't see what's
different today. Asking what it would take to branch is really the only
sensible thing I think we could possibly agree on.

> But David was reluctant for reasons that sound sensible. David, what would be 
> your opinion today?
> 
> Best,
> Jean
>

signature.asc
Description: This is a digitally signed message part