x11alpha seg fault / ESP ghostscript 7.05-5 / Linux 2.4 i386

[Sanjoy Mahajan]

PROBLEM
        gs, invoked by gv with antialiasing, segfaults when popping up
a zoom window.

SYSTEM INFO
       Pentium Linux 2.4.18 (RedHat 7.3), glibc 2.2.5, IBM Thinkpad
560, XFree86 4.2 libraries but XFree86 3.3.6 SVGA server, 8 bit color.

gs is ESP ghostscript 7.05-5 (from ghostscript-7.05.5-0.src.rpm).
gv is 3.5.8 from gv-3.5.8-18.7x RedHat 7.3 update RPM

LONGER VERSION  
       When viewing the file 'onepage.ps' with gv and -antialias and
then expanding a piece of the page (zoom factor 8), gives the error
message 'interpreter failed in zoom window'.  I didn't see a bug
report on bugs.ghostscript.com or see a relevant change in the version
7.06 changes file.

To trace the responsibility (gv or gs), I hacked 'gv' to duplicate in
a temporary file what it sends gs, to print the gs command line as it
invokes gs, and to print the GHOSTVIEW window id and its GHOSTVIEW and
GV_COLOR properties.

gv is sending postscript properly (`sent_to_gs.ps') and invokes gs
fine ("gs -dNOPLATFONTS -sDEVICE=x11alpha -dNOPAUSE -dQUIET -dSAFER
-"), but gs is dumping core near the end of the file, causing a broken
pipe, whence the 'interpreter failed' error.

The relevant files are in
<http://www.inference.phy.cam.ac.uk/sanjoy/gsbug/>

To reproduce the error just with a gs command requires setting up the
windows, which gv (an unhacked version) will do for you but you need
to know the window id's, for which 'xwininfo' and a little digging
will get you, or hack Ghostview.c in the source/ directory of the gv
distribution to print out the relevant info.  Details on that after
showing the results, using my hacked gv:

% /usr/src/redhat/BUILD/gv-3.5.8/source/gv onepage.ps -antialias &

debug_gs: GHOSTVIEW property [29360216]: 29360300 0 0 0 594 841 75.0252 75.0784 
0 0 0 0
debug_gs: GV_COLORS property [29360216]: Color 0 220
Starting [13406]: gs -dNOPLATFONTS -sDEVICE=x11alpha -dNOPAUSE -dQUIET -dSAFER 
- 

The 29360216 is the window id of the main ghostview window (or
whichever subwindow of it it gs should draw).  The GHOSTVIEW property
is "bpixmap orient llx lly urx ury xdpi ydpi [left bottom top right]"
as described in the gs.interface file in the gv documentation.

The postscript page is displayed fine with antialiasing.

Now I pop up a zoom window, in gv, by dragging with the middle button
using the word 'Glauber' (middle of the page, towards left) as the top
left corner, and going down four lines of text, about the width of
'Glauber', and zoom it by 8.

Then (the hacked version of) gv spits out:

debug_gs: GHOSTVIEW property [29360309]: 29360310 0 106 391 150 446 212.8 
212.143 106 391 444 395
debug_gs: GV_COLORS property [29360309]: Color 0 220
Starting [13407]: gs -dNOPLATFONTS -sDEVICE=x11alpha -dNOPAUSE -dQUIET -dSAFER 
- 

It creates a zoom window then fills it with nice antialias, magnified
text (note that the xdpi is now 212.8, rougly as it should be --
sqrt(8) times its value in the original window, 75.0252).  Then the
"GhostScript Messages" window shows "Error: PostScript interpreter
failed in zoom window", because gs has dumped core.

To reproduce the core dump, now that the zoom window is set up with
the bounding box coordinates, resolution, etc. in the GHOSTVIEW
property:

% env GHOSTVIEW=29360309 gs -dNOPLATFONTS -sDEVICE=x11alpha -dNOPAUSE -dQUIET 
-dSAFER - < sent_to_gs.ps
Segmentation fault

`sent_to_gs.ps' is a copy of what gv sends to gs (saved by my hacked
version of gv).  It's the file 'onepage.ps' with the beginning
comments removed and the trailer removed:

% diff sent_to_gs.ps onepage.ps
0a1,12
> %!PS-Adobe-2.0
> %%Creator: dvipsk 5.86 p1.5d Copyright 1996-2001 ASCII Corp.(address@hidden)
> %%based on dvipsk 5.86 Copyright 1999 Radical Eye Software 
> (www.radicaleye.com)
> %%Title: l99.dvi
> %%Pages: 1
> %%PageOrder: Ascend
> %%BoundingBox: 0 0 596 842
> %%DocumentFonts: CMSS10 CMSY10 CMCSC10 CMSL12 CMBX12 CMR10 CMMI10 CMBX10
> %%+ CMR8 CMMI8 CMSL10 CMTI12 CMTI10 CMR9 CMMI9 CMR6 CMSY9 CMR7 CMTT10
> %%+ CMTT9 CMEX10 CMBX9 CMBXTI10
> %%DocumentPaperSizes: a4
> %%EndComments
4115a4128,4131
> %%Trailer
> end
> userdict /end-hook known{end-hook}if
> %%EOF

I am reasonably sure that the problem is with gs and with its x11alpha
device, and not with gv or with the X window setup.  For example,
after getting the seg fault, I can still fill the zoom window with
non-antialiased, magnified text:

% env GHOSTVIEW=29360309 gs -dNOPLATFONTS -sDEVICE=x11 -dNOPAUSE -dQUIET 
-dSAFER - < sent_to_gs.ps

There's no error message, no seg fault, and the window fills with
non-antialiased text.

If you want to hack 'gv' to print out the window id and properties for
you -- in the Ghostview.c source file, add these lines near the end of
the Setup() function:

   fprintf (stderr, "debug_gs: GHOSTVIEW property [%ld]: %s\n",
            XtWindow(w), buf);
   fprintf (stderr, "debug_gs: GV_COLORS property [%ld]: %s\n",
            XtWindow(w), buf);

I've also put a Pentium binary (compiled with -mcpu=686) of my hacked
gv along with the other files.  It works fine alongside the vanilla
3.5.8 (redhat 3.5.8-18 rpm) config files.  If you use it, you'll see
that when gv starts gs the second time (for the zoom window), it
doesn't send all of `sent_to_gs.ps' -- the last 7K is left off because
gs segfaults and the pipe breaks, but gv is trying to send the whole
file (I checked that by tracing this gv using gdb).

-Sanjoy