Re: Do pretests reach end users? (was: When will emacs 27.1 be officially released?)

[Eli Zaretskii]

> From: Dmitry Alexandrov <dag@gnui.org>
> Cc: Kévin Le Gouguec <kevin.legouguec@gmail.com>,
>   liwei.ma@gmail.com,
>   emacs-devel@gnu.org
> Date: Fri, 03 Jul 2020 03:48:02 +0300
> 
> May I chime in?

Sure.

> Eli Zaretskii <eliz@gnu.org> wrote:
> > it takes a long time for Emacs problems to be discovered and reported.  A 
> > new Emacs release can take years to reach end users.  We are routinely 
> > receiving bug reports about changes made two or more releases ago.  If you 
> > are looking for a single most important reason why it takes so long to put 
> > out another pretest, it is this one: experience shows that it takes weeks 
> > if not months for enough people to try a pretest and report the problems 
> > they see.
> 
> Emacs pretest release?  Is that a thing for systems other than Microsoft 
> Windows and Guix?

First, I didn't say "pretest release": there's no such thing.  A
pretest is not a "release", it's a tarball meant to be tested by
people who track the Emacs development.  Its difference from the
corresponding Git branch is that it is a tarball that builds like a
release would, and thus one of its main purposes is to see that the
tarball itself doesn't miss anything (which would mean we need to fix
our procedure for producing the tarball).  And the other important
purpose is to catch the attention of people here and encourage them to
switch to the pretest version instead of the Git version, so that they
could discover any last-minute problems.

> — Unstable Debian comes with Emacs 26, Ubuntu gets Emacs from Debian.
> — Arch comes with version 26, AURʼs ‘emacs27’ is 27.0.60 — an outdated 
> February snapshot [1].
> — Unstable Nix, again, contain 26th Emacs.
> — Fedora provides only stable version of Emacs.
> — Homebrew — guess what?
> 
> And even pretest releases for Windows do not seem to be mentioned in any way 
> on <https://gnu.org/s/emacs>.  How users are supposed to get them?

They aren't.  We cannot control the policies of the various distros,
and cannot rely on them for being part of the pretest.  Sometimes they
are, nonetheless, presumably because the people who are responsible
for the distros read the announcements about the pretests, and that is
good.  But it isn't the main means for pretesting.

> In other words, I am not much surprised of the lack of early bugreports.

IME, changes to the Git repository also take several weeks to generate
reports about problems.  Severe problems, like crashes upon startup,
are, of course reported much faster, but a pretest version is always
quite stable and devoid of such problems.

So I don't see how all this could help making a release faster.  The
fundamental problem, as I see it, is that Emacs is a very large and
complex package, which can be used for a plethora of different jobs,
and it takes time for some of the features to be actually used by
someone, and yet more time for such usage to bump into some rare bug.
But rare bugs could nevertheless be severe enough for us to avoid.
How to solve this is the important question in this thread, at least
for me.