Re: [Texmacs-dev] General efficiency of TeXmacs

[Gubinelli Massimiliano]

Hi,

On 15 juin 09, at 00:24, Alex D wrote:

Actually, I spent a few hours hacking the cairo renderer to work with the Qt gui on Linux. When I finally got it to work, the TeXmacs windows would be all gray and the white page with text would blink into existence momentarily when a key was pressed but then the screen would turn gray again. I could'nt keep Qt from overwriting the drawn image with gray after every repaint and gave up.

I've tested the renderer mainly on Mac, indeed there are still problems on X11. For the moment you should configure with --with-cairo=linked. The rendering problems are probably due to some double-buffering misbehaviour (X11 does not have double-buffering so Qt is emulating it but we write direcly on the X11 surface bypassing the Qt double-buffering and everything goes wrong). 

The cairo renderer has at least two advantages over the plain Qt renderer: 

 * can render image masks (so that colored bimap glyphs text do not need to be cached once for each different color)

 * font drawing API is low level so we have direct access to the glyphs

You should be aware that in the current Qt port the mechanisms that blocks redrawing is user input happens is disabled. The X11 port is more sophisticate in this matter and I suggest you to look at that code to see how far we are in the Qt port.

I don't fully understand. You're saying all the full-windows redraws are not an error? I assume only only having the one partial redraw after a key press is the correct behavior.

I'm not sure: I think the typesetting engine send a series of redrawing messages: starting from a local one and going towards a global redrawing, but I'm not really sure. It should be checked on the X11 backend.

I'm very interested in that. How did you implemented native rendering of fonts in Qt? I've spent a lot of time trying to do this in Qt/Mac and while I've managed to do this it is not really straigforward (Mac cannot read Type1 fonts directly). Can you sent me your patches?

I'm using QFontDatabase::addApplicationFont() to register fonts with Qt. Then I use setPixelSize() to set the point size. I guess this won't work correctly with font's that have multiple optical sizes like Computer Modern, but personally I think Computer Modern is a terrible typeface, and should be replaced with Times as the default.

Qt Linux can handle type1 (.pfb) fonts just fine, so it's not a problem for me. The font files should be renecoded as OpenType and TrueType to ensure that they work on all platforms.

For the mac I wrote a routine (together with Jonatahn Kew of XeTeX) which convert the pfb font into the otf format supported by the Mac. To access direcly the glyphs you also need to find out the font encoding vector.

Reencoding fonts iself is easy. With FontForge I was able to reencode and merge the GPL'd Base 35 fonts (Times, Helvetica, Palatino, etc), Liberation, and DejaVu into TeXmacs' format.

apriori reencoding of the fonts could be a solution. However it would be nice to be able to use generic fonts (for example all the fonts already installed on the platform). Moreover encoding could be found out using freetype.

Another probles is that there are few (none?) OpenType fonts with proper support of mathematics. How do you handle mathematical glyphs in your home-brewed fonts?

TeXmacs uses FreeFont internally now to rasterize glyphs to bitmaps, so OpenType fonts will not pose a problem for the X11 port. But we will also need to have the fonts in pfa or type42 format so they can be embeded in postscript.

The only inefficiency with native Qt font rendering is that Qt has to do complicated script anlasis, shaping, deal with kashidas, etc for every character draw operation. But I don't think this is a big deal.

I do not like very much the Qt API for fonts. In my opinion it is too much high level for the needs of TeXmacs. this is why I wrote a cairo renderer.

I can certainly work on a patch for this, if you believe this is the way to go.

I'm sending you a patch which reorganize the way fonts talks to renderer. Its aim is to have an higher lever API so that native rendering of fonts can be properly implemented.

Virtual fonts still handle rendering via bitmaps (even if we should think to a cleaner solution via higher level commands sent to the renderer, other possibility: OpenType "glyplets").

I'm really interested in the problem of native font rendering. We should try to go on in this direction.

Best,

Massimiliano

PS: patch attached below

expfonts.patch
Description: Binary data

On Sun, Jun 14, 2009 at 2:36 PM, Gubinelli Massimiliano <address@hidden> wrote:

Hi,

On 14 juin 09, at 20:34, Alex D wrote:

Hi,

I've been experimenting with the Qt port lately on Linux. I'd just like to mention I get much better performance with -graphicssystem raster in case people are not aware of this option.

There is a cairo renderer available in the repo. We should experiment which one is faster (Qt or Cairo).

I've briefly profiled the rendering code, and noticed TeXmacs does three complete repaints and one partial for, essentially, every key press. It also seems to rebuild the main menu after every key press and this causes Qt to refllow of the whole window.

You should be aware that in the current Qt port the mechanisms that blocks redrawing is user input happens is disabled. The X11 port is more sophisticate in this matter and I suggest you to look at that code to see how far we are in the Qt port.

I've actually went in and disabled the extra whole-window repaints so only the partial repaint was done, and it wasn't rendering correctly :-(.

Also, I implemented native font rendering with Qt, which was quite easy. I think this is the way to go. The result looks much better and should also be quite a bit more efficient than the present implementation.

I'm very interested in that. How did you implemented native rendering of fonts in Qt? I've spent a lot of time trying to do this in Qt/Mac and while I've managed to do this it is not really straigforward (Mac cannot read Type1 fonts directly). Can you sent me your patches?

Best,

Massimiliano

 

Sincerely,

Aleksandr Dobkin

On Sun, Jun 14, 2009 at 11:11 AM, Norbert Nemec <address@hidden> wrote:

Hi there,

after doing a bit more profiling, I am beginning to realize just *how* inefficient some of the central C++ code in TeXmacs is:

a) For the central data types (string, tree, etc), nearly all routines should be defined as "inline" in the header files. The cleanest and simplest solution that I see would be to introduce additional files like string.inc.hpp that contain many of the routines that are currently defined in string.cpp and include this at the end of string.hpp. That way, the header file would not become cluttered, but the compiler could still do heavy inlining on these routines.

b) Profiling indicates that the program spends a tremendous amount of time on reference counting and deallocation. Moving to a garbage collector would significantly simplify the code and should result in quite some speedup. Especially for interactive programs, garbage collection is the fastest memory management strategy available since all the collecting is simply happening in idle time and the time critical code does not need to worry about deallocation at all.

c) The string class is a significant bottleneck. Every time a (char*) is converted, the data is copied to a newly allocated location. In most cases, this could be avoided by simply copying a pointer. This would mean that the string_rep class would need to carry a flag on whether the space is "owned" (and should be deallocated) or just "borrowed". I certainly is a non-trivial task to sort this out, but I believe it would be the single-most-rewarding detail in optimizing for performance.

... just my analysis for the moment. I know that it would mean significant work to work on any of this...

Greetings,
Norbert