Steve: Adding you because it's about MELPA and package archive discoverability, which you've probably thought about waaaay more about than I have. =) I think the newbie flow is that people usually go from MELPA -> figure out package-archives rather than starting with package-archives and finding MELPA. The code from MELPA's Getting Started guide might be one of the first things a newbie wants to add to init.el. Would you consider adding or linking to some kind of tutorial on editing init.el?
Other parts of the discussion follow:
There are very few package archives that meet our moral criteria
for suggesting their use. In fact, currently only one: GNU ELPA.
We are thinking of making another, which we can informally call NonGNU
ELPA. Some of the packages now in other archives could be included
there. The details remain to be worked out.
For moral reasons, we will not suggest an archive that contains
packages that steer or lead users towards installing or running
nonfree software. That is a brief summary of the rule in node
References in https://www.gnu.org/prep/standards/.
That's cool, GNU Emacs is a GNU flagship project after all, and lots of people learn about and become committed to free software philosophy because of it. That's good. So if MELPA doesn't make the cut, maybe the NonGNU ELPA could have enough popular packages so that people can install them easily and get the hang of things. Let's see what the top MELPA packages are based on downloads... Would things like Magit, Helm, and Flycheck be eligible for inclusion in the new package archive?
Let's think through a bunch of scenarios where package management hiccups might come up. How do people come to Emacs? From reading the newbie threads in
reddit.com/r/emacs, I get the sense that lots of people come in because they've seen a cool video or blog post about programming, Spacemacs, or Org Mode, or because they're Vim/VSCode users who are curious about the other side.
Org Mode newbies tend to spend some time familiarizing themselves with basic Org before digging into additional modules. The thing that usually trips people up is probably upgrading Org from an Emacs that hasn't opened an .org file before, but people seem to figure that out with some help. There used to be some confusion due to major differences between versions, but that seemed to have settled down after people got used to Emacs 26.1.
If newbies are coming in because of a Spacemacs programming video or because they're coming from Vim, they'll probably go straight to Spacemacs, which will solve the package management issue for them for a while. In fact, Spacemacs wants people to set the dotspacemacs-additional-packages variable and removes packages installed with package-install, so package management improvements in vanilla Emacs are irrelevant. Some people eventually leave Spacemacs for vanilla Emacs or something like Doom Emacs because they want something lighter-weight, but they'll probably be a bit more comfortable with Emacs by then.
So let's say they're starting with vanilla Emacs instead of Spacemacs. Most people's questions tend to show that they already know about MELPA and are running into problems setting it up, although there are a couple of questions from people who want to install a package that isn't on ELPA and they don't know about MELPA yet. I wonder what kind of hiccups people usually run into... Here are snippets from an IRC conversation in Feb 2020: "I want to install julia mode (for programming in julia) on my ubuntu 16.04 box. Apparently I should use the MELPA repository but looking at
https://melpa.org/#/getting-started is very intimidating" . The person turned out to have a problem with having both .emacs and .emacs.d/init.el and also having Emacs 24, and eventually got it working after upgrading. Another conversation from 2019-07-10 turned out to be a problem with editing .init.el instead of init.el (note initial .). Some people got tripped up by using add-to-list 'package-archives before package.el is loaded. The MELPA instructions for getting started have (require 'package) at the beginning, but the one for melpa-stable might confuse some people because it doesn't include that, and sometimes newbies don't notice the (require 'package) when copying parts of other people's configs. Most people seem to be okay with copying the right code, they just get tripped up by where to put it. Other questions about package archives tend to be related to servers being down, HTTP vs HTTPS, and so on.
Then of course there's figuring out which packages to install and how they work together, how to configure them, how to load them... People often ask for package/config recommendations on
reddit.com/r/emacs and community members share lots of resources, so I think that's something we can handle on the community side.
We probably lose a bunch of potential users in that gap between "Okay, I've installed Emacs" to "I've installed the packages I'm curious about" - something along the lines of "Why is it so hard to install ____? Are things going to be as frustrating as this?" Starter kits like Spacemacs do an interesting job of bridging that, although I think adoption is still a bit low. (Maybe 5-10%, as a totally wild guess?) Anyway, once they get past that gap and start having their first wins, they have a bit more incentive to stick around, as Emacs becomes more useful, more personalized, and more fun.
I don't think MELPA would change the getting started instructions to use Customize, so maybe tweaking the customization flow for package-archives is actually less relevant. If more packages (especially the more popular ones) could be installed without mucking around with package-archives or ~/.config/emacs/init.el (I need to get used to the new location!), that's probably a bigger win than nudging people to customize package-archives would be.
Hmm... Actually, come to think of it, where *do* we help people figure out how to tweak their init.el? Emacs tends to recommend Customize because it offers more guidance (and blindly copying Emacs Lisp into your config can lead to Big Problems), but people who write about Emacs or share their configs tend to post (setq ...)-type stuff (or use-package, which is becoming quite popular), so newbies want to edit their init.el and then run into all these problems finding the right file and figuring out what to do. There's (emacs) Init File, which is under Advanced Features > Customization. Should we think about init.el as something beginners might want to know about early? Maybe a mention or info link in the tutorial? Maybe a splash screen or Options menu shortcut to edit the init file so that at least they're editing the right place, and maybe even some code to check if the init file is getting shadowed? Maybe a M-x find-user-init-file or edit-user-init-file? It seems a little dangerous to nudge people towards init.el early, but newbies are already jumping into it and getting lost, so maybe we can make it easier for them and frame it as a natural part of working with Emacs. Come to think of it, tutorial writers are going to have to figure out how to talk about editing either ~/.config/emacs/init.el or ~/.emacs.d/init.el . If there's going to be a transition period for that anyway, it might make sense to promote something like M-x find-user-init-file as well.
Along those lines, maybe there could be a hello-world sort of package that people can install as part of the tutorial (or a tutorial part 2) that just helps them confirm that they can do that, although we might need a link to more troubleshooting info if they run into problems with proxies or other stuff. Something for the info manual, maybe?
Sacha