Re: Rethinking the design of xwidgets

[Akira Kyle]

On Thu, Oct 15, 2020 at 06:48 AM, Arthur Miller 
<arthur.miller@live.com> wrote:

> > As someone who has done a little bit of GPU programming, I 
> > think Emacs should
> > stay as far away from needing to care about the architecture 
> > and optimization
> > for GPUs as possible. It is hard and very non-portable.
> AFAIK OpenGL runs on more systems then Emacs, and is considered 
> as
> probably THE easiest graphics API out there (Turtle graphics 
> probably
> being easier :-)).
>
> It is superseeded by Vulkan, but Vulkan is probably too much for 
> an
> application like Emacs in terms of complexity vs benefit. 

Just a disclaimer that I haven't actually worked with OpenGL, my 
gpu experience was centered to compute instead of graphics tasks, 
so take everything I say with that grain of salt. Just because 
OpenGL is very cross platform, doesn't means its the preferred 
graphics API on a given platform. Higher level graphics APIs will 
often target some subset of OpenGL, Vulcan, Metal, and Direct3D 
depending on what platforms they want to support and what gives 
them the best performance.  I think Apple would love to kick 
opengl out of its ecosystem and force everyone to use Metal and 
AFAIK already has done so on iOS since they now control the 
physical GPU implementation with their own custom 
silicon. Furthermore, OpenGL being a low level graphics API, 
doesn't try to make decisions for you about the most efficient way 
to render what you have. Depending on the size and number of 
objects you want to draw and the resolution of the screen you want 
them draw on, some task will be better done on the CPU while 
others will be better served on a GPU. I don't think Emacs should 
be trying to figure that out since it can be difficult to do right 
and requires good performance benchmarking abilities.

> > I should have clarified
> > in my prior email that I really meant that Emacs could take 
> > advantage of newer
> > hardware like GPUs and other SIMD vectorization units not 
> > directly because it is
> > a huge PITA to work with, but through appropriate high level 
> > libraries. I don't
> > think Emacs should be in the business of trying to actually 
> > draw primitives like
> > lines and rectangles onto the glass directly whether through 
> > the frambuffer or
> > low level gpu primitives like opengl. It requires specific 
> > expertise to get
> > right and do efficiently that I think is beyond the scope of 
> > Emacs development.
> >
> > Since Emacs already has its own internal abstraction for 
> > actually doing the
> > drawing through the redisplay_interface, my point was more that 
> > Emacs should
> > gain hardware acceleration by ensuring the GUI toolkits it uses 
> > take advantage
> > of such features. I think gtk 4 will have even more integration 
> > with hardware
> > acceleration,
> In my world, Emacs would be better shying away from big GUI 
> toolkits and
> ginormous dependencies.
>
> Emacs window is (logically) one giant glorified tty in 
> principle; tucked
> into an OS window and Emacs does it's own redisplay of the major 
> os
> window area, so GUI toolkits are really just placeholders for 
> Emacs to
> render. Buttons, menus, toolbars can all be implemented by Emacs 
> buffers
> rendered as Frames, or Windows. Does not blend very well into 
> Gnome or
> KDE world, but with some styling might be pretty too look at 
> anyway.
>
> As I udnerstand Emacs now does it's own
> font and text rendering via Freetype , so question is if needs 
> Gtk at
> all more then being pretty on Gnome desktops? Maybe Cairo alone 
> is not
> bad, since it can be used as graphics API to render to and there 
> are
> also hardware accelerated versions via opengl bindings (look in 
> Mesa source).

My understanding of Emacs rendering is that it does rely on GUI 
toolkits more that you may think. I think the NS code renders via 
macOS's native font rendering, while on when on X --with-cairo, 
Emacs uses Freetype to render text. Perhaps someone may decide to 
implement text rendering using Pango to be more inline with the 
way GTK renders text.

Since Emacs doesn't even handle drawing its own rectangles but 
rather lets each toolkit draw them it its own native way I'm not 
sure it makes sense to move Emacs towards doing even more low 
level graphics stuff then it already does. In principle I think 
one could implement a EGL redisplay_interface which would then run 
across platforms, however it would potentially be a lot more code 
to implement everything from such level drawing primitives than 
the using the toolkits and it would loose the "native look" that 
the current, higher level approach maintains. I think the mac 
people tend to think their native text rendering is superior so it 
might be a hard sell to try to move Emacs to a cross-platform 
"rendered in-house" model. I think it would be really neat to 
explore such rendering backends for Emacs, I'm just not sure how 
much it would simplify things or bring tangible speed 
improvements. The webrender backend that someone made in remacs is 
in this vein, although I think even webrender is a much higher 
level graphics API since it sits atop OpenGL.

[1] https://github.com/remacs/remacs/pull/1581