Re: Why is Elisp slow?

[Ergus]

Hi Paul:

On Sun, May 05, 2019 at 03:25:59PM +1000, Paul W. Rankin wrote:
> I don't understand this sentiment. Is there some emergency in Emacs 
> development that I'm unaware of? Development seems faster and stronger 
> than it ever has in the decade or so I've been using Emacs -- there's 
> a new major release every year or so, and with the addition of 
> package.el there's the introduction of third-party package archives 
> and the explosion that is MELPA.
I agree with you in that development is still active. But the grown in
number of users and developers is pretty constant, while the number of
developers and computer users in the world has grow exponentially. But
also there are sections and full parts in the emacs code without expert
to maintain it right now.

Mane old developers has left and deep/infrastructure/architectural big
changes are every time more difficult to do and the tendency goes worst
with time (more and more code).

Just give a look to the rating of the other editors I
mentioned, and their git repository. In fact in the last year, the
articles recommending text editors for programmers rarely include Emacs
while vim is in all of them. Also in my work there are more
than 400 programmers and I am the only Emacs user. Same when I go to
conferences around. So this is a bad statistical signal. Without users
there are not potential contributor.

We don't have a technical problem because emacs is more powerful than
most of the editors in the lists around, but we have a social problem.

Package.el same as modules are the best and more important steps emacs
has made in the last years. It has motivated and organized many things
that were very problematic before, which is very important, but it is
something the GNU/Linux distros figured out like 25 years ago
(aptitude, yast, pacman, synaptic), so we were (and are) late (not
wrong, just late, delayed in time for the decisions).

Just looking at the number of code reviewers we have, it is a reason to
worry. We have much less developers/contributors in emacs than what vim
has these days, and at the same time we have more than 2 times more code
to maintain with more languages and functionalities. 

> The possibility of the current Emacs developers all dying out is not 
> something anyone needs to worry about for at least another 30-40 
> years, and I'm pretty sure we (the human race) will have vastly 
> different concerns by then.

Not only dying, but also leaving, getting busy with the years, or
keeping old standards and paradigms, workflows, tools, concerns... and
this is reflected automatically in the project. Emacs needs new blood,
new people with new needs, ideas and concerns.

> But even if we reach a point where there 
> is no one left maintaining/developing/using Emacs, it's open source 
> code... someone will discover it and if they find it useful or 
> interesting, they will continue developing it. Open source code can 
> never really die.

The idea (and our reponsability) is not to reach there if we can avoid
that. Otherwise we failed as maintainers/developers. 

> Where does this fear come from that an open source project will die if 
> it doesn't keep changing?

Is not keep changing, is keep usefulness for its main purpose. What
emacs does as main purpose "edit text" will always gonna be useful. But,
are we being "competitive" in this field? The emacs infrastructure and
philosophy has all the needed for that (and much much muuuuuch more),
but are we using it properly for that purpose?

I think it is the best infrastructure to edit text, but right now we
are not providing the best editor (which should be the main goal). And
we add more and more and more functionalities and code. It is common
to heard that "emacs is a good OS, it only lacks a good editor."
which makes me very angry, but from the current editors standards
point of view it is right.

We don't need to keed changing, we need to do the right changes for
the project long path. FOr example: provide interaction with other
systems, languajes, optimize the compiler or rely in some other one
that already do it well and we trust in, cleanup code and portions of
code not fundamental for the emacs functionality or move them to
elpa/melpa. Standarize a bit more the bindings and function names.

For example the performance issues many people complain about
frequently is just a consequence of the extreme generalization in
functionalities and overheads in the infrastructure added for doing
"everything", but many of those things we barely do well in 2019
standards (some of then were added, and abandoned), so most users use
something different for that (like web browsing, reading PDF, play
tetris, play music, create buffers with buttons, have duality for TUI
and GUI in the same program, support very old architectures that are
inexistent these days) And all this is more and more and more code to
maintain, and more fields of expertise needed in our domain.

Same happens with development, we have files with more than 33000 lines
that we need to maintain, and any change in it is veeeeery complicated
and complex, but most of the code is to give support to some
functionalities very rarely used in practice.

And the infrastructure itself is poorly documented and complex
(specially the C interaction with Lisp from within emacs files.)
Thanks Eli, Stefan and the others who takes care of new users like me,
but that method to introduce developers is not scalable and makes them
to spend a lot of time.

> If a possible contributor has cloned a project repository to their own 
> machine and has made some changes, the fork-based workflow requires 
> that they: create an account at the origin project's GitHub or GitLab 
> (or the project's GitLab instance), create a forked repository there, 
> add the fork as a remote on their machine, push the changes, then open 
> a pull request.
>
> Once you use a git send-email workflow, this fork-based workflow will 
> seem convoluted and unnecessarily centralised. All a contributor need 
> do is clone the project repository, commit some changes then run:
>
>   git send-email HEAD^
>
> And send the email to the project's owner/mailing list. No account 
> creation necessary. Check out:
>
>   https://git-send-email.io/
>
> Yes the copyright assignment procedure is a deterrent to contributing 
> to Emacs/ELPA; this discussion is probably for another thread.
>
> --
> https://www.paulwrankin.com

This is not better than having a fork-merge request workflow. But also
forces to handmake a lot of the steps the new interfaces do for you like
reading the patch and make comments, follow those comments and reply to
specific parts of them, make then visible in the web, automatic
test/benchmark the functionalities, changes, close the requests or deny
them, give access for all that to the users in the future. Group people
in different forks working in the same functionality, bug, issue,
extension; test the code in the fork head automatically before merging
it in master. Is not impossible to do all this, it is just more manual,
time consumming and unfamiliar for 90% of the developers I know.

But also it is somehow dangerous, because actually everybody writes in
the master branch, while in my opinion only the managers (Eli, Stefan,
Alan...) should write there and the rest (me) should write only in the
forks and make pull requests (similar to what we do with branches, but
safer).

Now for the project it looks like:

bug: bugtracker
developement: savannah
developer interaction: mailing list
user interaction: reddit + help mailing list
documentation: within emacs
project main page: gnu.org
contribution: copyright process+mailing list+savannah
questions: another mailing list
packages: elpa
more packages: melpa
user hacks and advices: emacswiki