Introducing emacs-webkit and more thoughts on Emacs rendering (was Rethinking the design of xwidgets)

[Akira Kyle]

Awhile ago I started a thread on rethinking the design of 
xwidgets, there was a lot of good discussion about the future of 
Emacs' rendering capabilities. It personally motivated me to try 
to better understand how Emacs' redisplay works and just overall 
better understand the C core of Emacs. As part of that exercise, I 
decided to see if I could port the existing xwidgets code into a 
dynamic module. The only xwidget Emacs currently implements is a 
webkit widget, so this effort turned into creating a dynamic 
module for embedding webkit in Emacs. It also happens that I've 
been wishing that I could never leave Emacs and a general web 
browser was the last daily app that I begrudgingly used outside of 
Emacs. Anyways here's the result: 
https://github.com/akirakyle/emacs-webkit.git (sorry for the 
github hosting, I just have yet to make the jump to another 
platform -- I guess what they say about vendor lock-in is true).

I was able to implement all of the current features of the webkit 
xwidget plus some more. I think having this as a dynamic module 
makes it feel less hacked into Emacs (although still a bit hackey 
in some places). One interesting additional feature I was able to 
implement was support for TUI emacs. Essentially it opens a 
dedicated window for displaying webkit which can be controlled 
through the keybindings in the emacs buffer corresponding to that 
webkit view (of course one needs to have running a running 
graphical session pointed to in $DISPLAY). Perhaps when this code 
stabilizes, it might make sense to remove the experimental xwidget 
support from Emacs if this is deemed a worthy successor to such 
features.

I also wanted to make a few more general comments prompted by the 
previous thread about the future of Emacs redisplay and rendering 
given what I've learned in this exercise. I now think its fairly 
foolish to try to integrate too much of a toolkit library's 
widgets into Emacs. There will always be this friction between 
Emacs and the toolkit library wanting to handle the business of 
drawing to the glass. In a lot of ways I've come away from this 
very impressed with how Emacs manages to have native toolkit ports 
to gtk, ns, win, and TUI rendering given the differences in how 
each wants to handle drawing to the screen, but on the other hand 
there's a lot of complexity and code behind making this happen, 
which presents a large barrier to entry to people like me who may 
want to hack on the lower level drawing capabilities of 
Emacs. We've also seen hardware change a lot since Emacs display 
machinery was written. CPUs have gotten faster and now everyone 
has a GPU. I think there's a potential for Emacs to become both 
snappier and more extensible with respect to its rendering. I had 
initially argued that Emacs probably shouldn't deal with rendering 
on the level of opengl, however I think as I've understood better 
the way Emacs currently handles rendering everything, that's 
exactly the direction Emacs should move in, *if it can*.

So here is my new pie-in-the-sky proposal for what the future of 
Emacs rendering could look like. Given the success and hopefully 
soon-to-be in master native-comp branch, perhaps it may not be so 
crazy to think about moving more rendering to elisp. If a clean 
elisp interface to opengl were defined and Emacs were able to 
render itself using this optimized elisp opengl interface, the low 
level code for each platform could be minimized to just providing 
the opengl surface and exposing input events. In fact there may be 
ways in which elisp is well suited to the way modern GPU graphics 
are rendered using scene graphs. In fact gtk4 is moving its 
rendering to such an approach in order to take advantage of 
hardware graphics acceleration. I think a first step would likely 
be to focus on implementing an elisp interface to cairo as Emacs 
is already using more of cairo to render itself. If it seems like 
the performance is good and elisp can represent graphical 
operations well, than that might be a good indication that such a 
model could be successful path forward towards an Emacs that could 
bring its extensibility to its own basic rendering.