Re: On elisp running native

[Adam Porter]

Hi Andrea,

Andrea Corallo <address@hidden> writes:

> - Finally I've set up a docker image for people willing to test it
>   without having to configure and compile Emacs and libgccjit.  Now that
>   I've learned how it works I plan to use it to setup a small CI to keep
>   an eye that the latest GCC does not break with our build.

Thank you for setting up the Docker image.  I had tried to build the
native-comp branch on my system but finally was unable to overcome a
weird error related to libgccjit (probably because I need to upgrade my
system).  I even tried using Guix to build it, but libgccjit isn't
packaged in it (and I'm completely new to Guix packaging, so I wasn't
able to figure out how to adjust the GCC build for that in a reasonable
amount of time).

I managed to get the Docker image working, and I did a little testing
with one of my packages, org-ql, which is for searching Org files.  I
installed it and its dependencies, then used native-comp-async to
compile all of ~/.emacs.d/elpa.  Then I restarted Emacs to ensure that
the native libraries were loaded, and finally I ran the following code
and observed a more than 2x improvement!

#+BEGIN_SRC elisp
  ;; Reset the cache to force search to run again.
  (setf org-ql-cache (make-hash-table :test #'equal))

  ;; Set my personal groups (omitted) and enable org-super-agenda-mode
  ;; to display in groups.
  (setf org-super-agenda-groups my-groups)
  (org-super-agenda-mode)

  ;; Load all my Org files and search them.
  (let* ((enable-local-variables nil)
         (files (directory-files org-directory 'full (rx ".org" eos))))
    (dolist (file files)
      (find-file-noselect file 'nowarn))
    (benchmark-run-compiled
        (org-ql-search files
          ;; The (tags) selector tends to be slow compared to other
          ;; predicates because of searching the hierarchy repeatedly
          ;; and tag inheritance, so it makes for an interesting test.
          '(and (todo) (tags "Emacs"))
          :super-groups org-super-agenda-groups)))
#+END_SRC

#+RESULTS:
| 5.819892666 | 1 | 0.6101534569999956 |  In Emacs 26.3
| 2.317804428 | 3 | 0.6223487619999997 |  In Emacs 28 with native-comp

Then I tested this more complex query and observed an even more
impressive speedup of about 3.4x!

  '(and (not (done))
        (or (habit)
            (deadline auto)
            (scheduled :to today)
            (ts-active :on today)))

#+RESULTS:
| 8.377843199 | 1 | 0.6277314330000081 |  In Emacs 26.3
| 2.436048375 | 2 | 0.4278436059999997 |  In Emacs 28 with native-comp

This is very exciting!  The difference between 8.4 seconds and 2.4
seconds is a huge usability improvement.

And everything seems to work fine!  No bugs, errors, or crashes of any
kind.

I did encounter a minor issue related to macro-expansion and loading
that may be my fault.  When I tried to (require 'org-ql) or call a
function that is autoloaded from it, I got an error saying that the .eln
file did not provide the org-ql feature.  I surmised that meant that the
file hadn't been compiled properly, so I looked at the async compilation
log and saw an error saying that the variable org-ql-predicates was not
defined.  At this URL you can see a macro definition,
org-ql--def-plain-query-fn, and the call to it that's wrapped in
cl-eval-when to ensure it works properly with the byte-compiler:

https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L1671

The macro uses the value of org-ql-predicates, which is defined here:

https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L126

But when the org-ql--defpred macro:

https://github.com/alphapapa/org-ql/blob/06481960764a55a36d886e1db79a58258d5d2ffb/org-ql.el#L140

...is called, it adds to that variable.  So all of that works with the
byte-compiler, but apparently not with the native compilation.

To work around it, I wrapped (defvar org-ql-predicates...) in
eval-and-compile, and then wrapped all of the calls to org-ql--defpred
in eval-and-compile as well.  Then I deleted the org-ql.eln file and
recompiled it, restarted Emacs, and then everything worked fine.

I'm guessing that this issue is "native" to the native-compilation and
might need to be documented in some way, but I wouldn't expect it to
affect many packages, only ones that do tricky things with variables and
macro-expansion like this.

So, a few ideas for fixes or improvements:

1.  I guess the .eln file should not have been produced or left behind
    in the directory, since there was a compilation error.
2.  The compilation error should probably be shown as a warning or
    something, because it's easily lost in the async compilation log
    buffer.
3.  It would probably be good if the loader fell back to .elc and .el
    files if the .eln file doesn't seem to work, and showed a warning
    when doing so.  The library would remain usable in its byte-compiled
    form, and in the case of a package like this, the user could then
    report the native-compilation error to the developer.

Other than that minor issue, everything works and feels very fast!  This
is great!

>   ~eln~ files are compiled in specific sub-folders taking in account
>   host architecture and current Emacs configuration to disambiguate
>   possible incompatibilities.

>  As example a file ~.../foo.el~ will compile into something like
>  ~.../x86_64-pc-linux-gnu-12383a81d4f33a87/foo.eln~.

FYI, I did not see the .eln files being put in subdirectories like that
in the Docker image.  Maybe it didn't receive that update yet.

This is all very exciting.  I'm imagining an update to package.el that
would byte-compile packages immediately, as usual, then start
native-compile-async'ing them in the background, loading each file's
native version as it completes, with the packages being immediately
usable and seeming to get faster as each file is compiled and loaded.

> I'm generally very satisfied (surprised) about the stability of the
> toy. I'm looking forward going into compile time mitigation.  I guess
> I'll start this weekend with deferred compilation and GCC profiling.

I think it isn't a toy anymore!  :)  Thanks very much for your work on
this.  Exciting times for Emacs, as this feature, in one form or
another, has been eagerly awaited for years.