Re: Why is Elisp slow?

[Ergus]

On Sun, May 05, 2019 at 11:50:16AM -0400, Stefan Monnier wrote:
> > The more I think about Guile Emacs and the Guile project itself, I start to
> > think that it???s not a problem of manpower, but something more
> > fundamental.
>
> Quite right.
>
> > Is it feasible (or even possible) to make two different high
> > level languages to interoperate in the engine level?????
>
> Language interoperation is very difficult, and even more so if you want
> both languages to be usable "equally" (as opposed to a high-level
> language with FFI bindings, say).  The .NET platform aims to do just
> that, and note that it's fairly complex and the most successful
> languages there were specifically designed/tweaked for that platform.
It looks more complex because the starting point is a High level
language as Lisp so any port will need many data types conversion and
copies with the corresponding overheads (and ugly code).

All the infrastructure is based in the Lisp Objects, types and data
structures and functions. But IF the base API were designed with basic C
types (int, doubles, char*) and structs, and language agnostic, the
interface with Lua, Python or Ruby will be pretty simple (even Fortran,
or anything else).

That's what vim did and they now support many languages for extending
and create plugins. (I know it is a different system and I am not saying
we have to move in that direction right now.)

But, instead of create lisp object from C the only we need it to create
an interface for the C functions that need access from Lisp and use them
in the interactive or high level lisp specific functions. But the low
level functions will be all exposed in the C library and we could access
then even with dlopen in some cases.

Of course I am not saying that we should develop code inside emacs core
with all the different languages or change language every 10 years. For
me it is fine Elisp (or common lisp even better if that reduces the
complexity of maintaining our on compiler and potentially improves
performance). But to give the user the possibility to create plugins; a
C API is much more useful and simple. And if we want to port some of
those to elisp will be also easier. I specially don't share the choice
of developing core parts (API) in Lisp directly because having them in C
is faster, simpler, more accessible from outside if needed and usually
gives less overheads to the starting time, and GC.

We already have the API with the modules, but if you see most of the
complexity comes from creating and converting lisp types to something
else and do callbacks to lisp functions, and parsing the arguments.

Defun or any other macro could be emulated if we have to, but again,
that's just a complexity of making everything Lisp centric instead of C
centric. 

Bringing a C api and use Lisp for high level or interactive functions
reduces overheads, simplify the C code we have and enables a more
modular design. But I know that try to do that now is not realistic with
the amount of code and people we have. So that's why I was talking about
Guile, but after all it seems not to be realistic either.

So whats the alternative? Keep our interpreter as is and deal with few
developers (and decreasing) and performance issues and a lot of code to
maintain?




> > For example, will it even make sense to have some Elisp macros in scheme
>
> Interoperation between "similar" languages like Elisp and Scheme can
> probably be made to be tolerable (e.g. I guess most Emacs-specific
> special forms of Elisp, such as `save-excursion`, could be provided on
> the Scheme side without too much trouble (famous last words)), but
> macros seem pretty hard to handle in general (i.e. will require manual
> work on a macro-by-macro case).
>
> > land, or even lua land, python land, JS land?????
>
> For these, macros aren't the only problem, since there's also the
> problem of the different kinds of datatypes used.  What would JS objects
> look like in Elisp, and what would Elisp objects look like in JS?
>
> > I would like a JSish API for emacs when writing packages, not a low
> > level wrapper around Elisp APIs.
>
> Indeed.
>
> > What would I do to use a Python module in JS????? Unless the language itself is
> > designed to be easily interoperable with elisp by superseding concepts (such
> > as Clojure superseding Java), I???m pretty sure many people won???t be satisfied
> > in any kind of implementation.
>
> Exactly.
>
>
>        Stefan