Crash in XRenderComputeTrapezoids()
Submitted by Trond Kjernasen
Assigned to Keith Packard @keithp
Description
I think I've found a bug in the Xrender library. The following little bit of code crashes for me on any of the Linux systems that I've tried it on (a compileable test program is available here: http://trolls.troll.no/trond/xrender_bug.tgz):
... case Expose: { XExposeEvent * ev = (XExposeEvent *) &event; XftColor xfc; const uint A = 127, R = 255, G = 255, B = 255; xfc.pixel = 0x0; xfc.color.alpha = (A | A << 8); xfc.color.red = (R | R << 8) * xfc.color.alpha / 0x10000; xfc.color.green = (B | G << 8) * xfc.color.alpha / 0x10000; xfc.color.blue = (B | B << 8) * xfc.color.alpha / 0x10000; Picture src = XftDrawSrcPicture(xft_hd, &xfc); Picture dst = XftDrawPicture(xft_hd); XPointDouble poly[4] = {{76.0, 29.0}, {69.0, 27.0}, {52.0, 25.0}, {52.0, 26.0}}; XRenderCompositeDoublePoly(dpy, PictOpOver, src, dst, XRenderFindStandardFormat(dpy, PictStandardARGB32), 0, 0, 0, 0, poly, 4, 1); } ...
GDB backtrace is here:
Program received signal SIGSEGV, Segmentation fault.
0x40254dd4 in XRenderComputeTrapezoids (edges=0x805a5f0, nedges=4, winding=1,
traps=0x806afe8) at Poly.c:197
197 traps->right = en->edge;
(gdb) bt
#0 0x40254dd4 in XRenderComputeTrapezoids (edges=0x805a5f0, nedges=4,
winding=1, traps=0x806afe8) at Poly.c:197
#1 0x402550ec in XRenderCompositeDoublePoly (dpy=0x804a008, op=3,
src=60817412, dst=60817413, maskFormat=0x8053d28, xSrc=0, ySrc=0, xDst=0,
yDst=0, fpoints=0xbfffe440, npoints=4, winding=1) at Poly.c:296
#2 0x08048acf in event_loop (dpy=0x804a008) at crash1.c:64
#3 0x08048b6d in main (argc=1, argv=0xbfffe5d4) at crash1.c:87 Valgrind output for here:
[snip] ==3959== Invalid write of size 4 ==3959== at 0x40484DA4: XRenderComputeTrapezoids (Poly.c:194) ==3959== by 0x404850EB: XRenderCompositeDoublePoly (Poly.c:296) ==3959== by 0x8048ACE: event_loop (crash1.c:64) ==3959== by 0x8048B6C: main (crash1.c:87) ==3959== Address 0x414D9F20 is 0 bytes after a block of size 768 alloc'd ==3959== at 0x4002ACB4: malloc (in /usr/lib/valgrind/vgskin_memcheck.so) ==3959== by 0x40484F07: XRenderCompositeDoublePoly (Poly.c:242) ==3959== by 0x8048ACE: event_loop (crash1.c:64) ==3959== by 0x8048B6C: main (crash1.c:87) [snip]
There is an out-of-bound write in the line above, and it seems that there is too little space allocated to "traps" (or really, the "edges" var) in certain situations.
I'd appreciate it if anyone can confirm that this is actually a bug. Thanks.
-- Regards, Trond K.