Re: [Texmacs-dev] General efficiency of TeXmacs

[Alex D]

I've tested the renderer mainly on Mac, indeed there are still problems on X11. For the moment you should configure with --with-cairo=linked. The rendering problems are probably due to some double-buffering misbehaviour (X11 does not have double-buffering so Qt is emulating it but we write direcly on the X11 surface bypassing the Qt double-buffering and everything goes wrong).

Thanks, this actually worked. Unfortunately, performance lags behind Qt -graphicssystem raster for me.

I looked at your patch. I couldn't find the definition of mac_native_draw_char(), but presumably this calls caro_show_glyphs? This is certainly an improvement, but were you able to get rid of the calls to tt_font_glyphs_rep::get (int i) from tex_font_rep::get_extents (string s, metric& ex), do you use cairo_glyph_extents()?

The cairo renderer has at least two advantages over the plain Qt renderer: 

 * can render image masks (so that colored bimap glyphs text do not need to be cached once for each different color)

 

* font drawing API is low level so we have direct access to the glyphs

But if we do decide to use Cairo or Qt, font glyph caching will be handled for us so we won't have to worry about it :-).

Although Cairo does support Quartz and Win32 fonts in addition to Freetype we would have to write platform specific code for Windows and Mac OS to get truly native font rendering. An advantage Qt font rendering is that it can automatically use Freetype, XLFD, or native Mac OS and Windows font drawing routines without us having to write any cross-platform code. This is the main advantage from my perspective.

You should be aware that in the current Qt port the mechanisms that blocks redrawing is user input happens is disabled. The X11 port is more sophisticate in this matter and I suggest you to look at that code to see how far we are in the Qt port.

I don't mean to be snarky, but according to my understanding, the only practical affect of this feature is that if you're editing a large document and press and hold the backspace key, it leads to some visual corruption onscreen and before you know it you've deleted eight paragraphs. At least with Qt TeXmacs, it deletes one character at a time (this way you can stop before you get too far!). Try this with Help->Full Manuls->User Manual.

Anyway, I'm attaching my patch for native font rendering in the Qt port. (also here: http://cs.wisc.edu/~aleksand/texmacs/qt-fonts-patch.txt)  I would appreciate any feedback. I've tried to make it as short as possible. The main painting code is quite simple:

qt_renderer_rep::draw (int c, font_glyphs fng, SI x, SI y) {
  if( fng->is_qt_font ) {
    tt_font_glyphs_rep *ttfng = static_cast<tt_font_glyphs_rep*>(fng.rep);
    int size = ttfng->size;
    int dpi = ttfng->dpi;
    int pixel_size = (size * dpi) / ( 72 * sfactor);
   
    QFont *font = &ttfng->qt_font;
    font->setPixelSize(pixel_size);
    painter.setFont(*font);
   
    decode(x, y);
    char str[2] = {c, '\0'};
   
    painter.drawText(x, y, str);
    return;
  }

Colors work correctly and no need for clipping.

I've tried to eliminate calls to tt_font_glyphs_rep::get (int i) because it means extra rendering has to be done in addition to what Qt is doing. To that end I've commented out some code in tex_font_rep::get_extents (string s, metric& ex) in Plugins/Metafont/tex_font.cpp. I mean I can kind-of understand why you would care about the advance values of glyphs when computing bounding box dimensinos but why is this only taken into account when figuring out (x3, y3), (x4, y4) but not (x1, y1), (x2, y2)? Are (x1, y1), (x2, y2) are supposed to be the actual dimensions, but (x3, y3), (x4, y4) are for drawing only? I'm really curios about this. And setting (x3, y3), (x4, y4) to even crazy values does not seem to yield any affect. [Nvervmind, this is only true on the Qt port, probably because of excessive repaints. But I sill don't uderstand the behavior on X11.].

PFB fonts work fine with Qt in Linux, but Mac OS only works with OpenType and TrueType. PFA or Type42 format files are needed for embedding into postscript.

One issue is that TeXmacs references fonts by file name but Qt does so by family name. The famly name for ecrm is European Computer Modern, for instance. Either one will have to change so that they match (The family name can also be obtained from FreeType when the font is loaded). For this reason, you will need my fonts package to see anything even on Linux.

I've reencoded several fonts into OpenType format but keept TeXmac's encoding to work with the new system. I've also included PFA or Type42 versions for embedding into postscript. The Times font has the best support. Latin is fully supported along with Greek for math. The fonts are derived from URW++ Nimbus Roman No9 for Latin, TX fonts for Greek, and Modern Computer for the math symbols. Kerning and ff, fi, fl, ffi, ffl ligatures are supported. I've replaced v and w in the math italic font with their curvy substitutes from TX fonts. There is regular, bold, bold-italic, small-caps, bold-smallcaps, slanted, and bold-slanted. All the math symbols are from Computer Modern (alternatively they can be taken from TX fonts). The license is GPL.

The package can be obtained here: http://cs.wisc.edu/~aleksand/texmacs/font-otf.tar.gz

To install it execute install.sh. Be aware that it will overwrite your personal my-init-texmacs.scm so back it up beforehand.

There are 63 fonts comprising 13 typefaces. All under a free license. In all they take up about 8MB.

Sincerely,

Aleksandr Dobkin

On Mon, Jun 15, 2009 at 4:52 AM, Gubinelli Massimiliano <address@hidden> wrote:

Hi,

On 15 juin 09, at 00:24, Alex D wrote:

Actually, I spent a few hours hacking the cairo renderer to work with the Qt gui on Linux. When I finally got it to work, the TeXmacs windows would be all gray and the white page with text would blink into existence momentarily when a key was pressed but then the screen would turn gray again. I could'nt keep Qt from overwriting the drawn image with gray after every repaint and gave up.

I've tested the renderer mainly on Mac, indeed there are still problems on X11. For the moment you should configure with --with-cairo=linked. The rendering problems are probably due to some double-buffering misbehaviour (X11 does not have double-buffering so Qt is emulating it but we write direcly on the X11 surface bypassing the Qt double-buffering and everything goes wrong). 

The cairo renderer has at least two advantages over the plain Qt renderer: 

 * can render image masks (so that colored bimap glyphs text do not need to be cached once for each different color)

 * font drawing API is low level so we have direct access to the glyphs

You should be aware that in the current Qt port the mechanisms that blocks redrawing is user input happens is disabled. The X11 port is more sophisticate in this matter and I suggest you to look at that code to see how far we are in the Qt port.

I don't fully understand. You're saying all the full-windows redraws are not an error? I assume only only having the one partial redraw after a key press is the correct behavior.

I'm not sure: I think the typesetting engine send a series of redrawing messages: starting from a local one and going towards a global redrawing, but I'm not really sure. It should be checked on the X11 backend.

I'm very interested in that. How did you implemented native rendering of fonts in Qt? I've spent a lot of time trying to do this in Qt/Mac and while I've managed to do this it is not really straigforward (Mac cannot read Type1 fonts directly). Can you sent me your patches?

I'm using QFontDatabase::addApplicationFont() to register fonts with Qt. Then I use setPixelSize() to set the point size. I guess this won't work correctly with font's that have multiple optical sizes like Computer Modern, but personally I think Computer Modern is a terrible typeface, and should be replaced with Times as the default.

Qt Linux can handle type1 (.pfb) fonts just fine, so it's not a problem for me. The font files should be renecoded as OpenType and TrueType to ensure that they work on all platforms.

For the mac I wrote a routine (together with Jonatahn Kew of XeTeX) which convert the pfb font into the otf format supported by the Mac. To access direcly the glyphs you also need to find out the font encoding vector.

Reencoding fonts iself is easy. With FontForge I was able to reencode and merge the GPL'd Base 35 fonts (Times, Helvetica, Palatino, etc), Liberation, and DejaVu into TeXmacs' format.

apriori reencoding of the fonts could be a solution. However it would be nice to be able to use generic fonts (for example all the fonts already installed on the platform). Moreover encoding could be found out using freetype.

Another probles is that there are few (none?) OpenType fonts with proper support of mathematics. How do you handle mathematical glyphs in your home-brewed fonts?

TeXmacs uses FreeFont internally now to rasterize glyphs to bitmaps, so OpenType fonts will not pose a problem for the X11 port. But we will also need to have the fonts in pfa or type42 format so they can be embeded in postscript.

The only inefficiency with native Qt font rendering is that Qt has to do complicated script anlasis, shaping, deal with kashidas, etc for every character draw operation. But I don't think this is a big deal.

I do not like very much the Qt API for fonts. In my opinion it is too much high level for the needs of TeXmacs. this is why I wrote a cairo renderer.

I can certainly work on a patch for this, if you believe this is the way to go.

I'm sending you a patch which reorganize the way fonts talks to renderer. Its aim is to have an higher lever API so that native rendering of fonts can be properly implemented.

Virtual fonts still handle rendering via bitmaps (even if we should think to a cleaner solution via higher level commands sent to the renderer, other possibility: OpenType "glyplets").

I'm really interested in the problem of native font rendering. We should try to go on in this direction.

Best,

Massimiliano

PS: patch attached below

On Sun, Jun 14, 2009 at 2:36 PM, Gubinelli Massimiliano <address@hidden> wrote:

Hi,

On 14 juin 09, at 20:34, Alex D wrote:

Hi,

I've been experimenting with the Qt port lately on Linux. I'd just like to mention I get much better performance with -graphicssystem raster in case people are not aware of this option.

There is a cairo renderer available in the repo. We should experiment which one is faster (Qt or Cairo).

I've briefly profiled the rendering code, and noticed TeXmacs does three complete repaints and one partial for, essentially, every key press. It also seems to rebuild the main menu after every key press and this causes Qt to refllow of the whole window.

You should be aware that in the current Qt port the mechanisms that blocks redrawing is user input happens is disabled. The X11 port is more sophisticate in this matter and I suggest you to look at that code to see how far we are in the Qt port.

I've actually went in and disabled the extra whole-window repaints so only the partial repaint was done, and it wasn't rendering correctly :-(.

Also, I implemented native font rendering with Qt, which was quite easy. I think this is the way to go. The result looks much better and should also be quite a bit more efficient than the present implementation.

I'm very interested in that. How did you implemented native rendering of fonts in Qt? I've spent a lot of time trying to do this in Qt/Mac and while I've managed to do this it is not really straigforward (Mac cannot read Type1 fonts directly). Can you sent me your patches?

Best,

Massimiliano

 

Sincerely,

Aleksandr Dobkin

On Sun, Jun 14, 2009 at 11:11 AM, Norbert Nemec <address@hidden> wrote:

Hi there,

after doing a bit more profiling, I am beginning to realize just *how* inefficient some of the central C++ code in TeXmacs is:

a) For the central data types (string, tree, etc), nearly all routines should be defined as "inline" in the header files. The cleanest and simplest solution that I see would be to introduce additional files like string.inc.hpp that contain many of the routines that are currently defined in string.cpp and include this at the end of string.hpp. That way, the header file would not become cluttered, but the compiler could still do heavy inlining on these routines.

b) Profiling indicates that the program spends a tremendous amount of time on reference counting and deallocation. Moving to a garbage collector would significantly simplify the code and should result in quite some speedup. Especially for interactive programs, garbage collection is the fastest memory management strategy available since all the collecting is simply happening in idle time and the time critical code does not need to worry about deallocation at all.

c) The string class is a significant bottleneck. Every time a (char*) is converted, the data is copied to a newly allocated location. In most cases, this could be avoided by simply copying a pointer. This would mean that the string_rep class would need to carry a flag on whether the space is "owned" (and should be deallocated) or just "borrowed". I certainly is a non-trivial task to sort this out, but I believe it would be the single-most-rewarding detail in optimizing for performance.

... just my analysis for the moment. I know that it would mean significant work to work on any of this...

Greetings,
Norbert

_______________________________________________
Texmacs-dev mailing list
address@hidden
http://lists.gnu.org/mailman/listinfo/texmacs-dev

diffs
Description: Binary data