Bug 11503 - Xserver 1.3 crash in fbPolyline() when using xpdf
Summary: Xserver 1.3 crash in fbPolyline() when using xpdf
Status: RESOLVED FIXED
Alias: None
Product: xorg
Classification: Unclassified
Component: Server/General (show other bugs)
Version: 7.2 (2007.02)
Hardware: Other All
: medium normal
Assignee: Xorg Project Team
QA Contact: Xorg Project Team
URL: http://bugs.debian.org/cgi-bin/bugrep...
Whiteboard:
Keywords:
Depends on:
Blocks: xserver-1.9
  Show dependency treegraph
 
Reported: 2007-07-09 00:52 UTC by Brice Goglin
Modified: 2010-08-20 10:14 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
0001-fb-Fix-overflow-computation-in-lines-arcs-polys-glyp.patch (917 bytes, patch)
2010-08-17 12:27 UTC, Adam Jackson
no flags Details | Splinter Review
11503.patch (1.56 KB, patch)
2010-08-19 13:21 UTC, Adam Jackson
no flags Details | Splinter Review
Creates window with visible coordinates > 32767 and attempts to draw (1.07 KB, text/plain)
2010-08-20 09:58 UTC, Keith Packard
no flags Details

Description Brice Goglin 2007-07-09 00:52:24 UTC
Bug reported by Albert Cahalan in the Debian BTS 2 years ago, still applies and easy to reproduce (here on i386 and by the submitter on powerpc).

Take the PDF file that have been saved in the Debian BTS:
http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=59;filename=chapter64.pdf;att=1;bug=320627

Open it with *xpdf* and play with the index scrollbar on the left, the server will crash soon. Here's a backtrace (from Debian Xserver 2:1.3.0.0.dfsg-7):

(gdb) bt full
#0  0xb7ab6ad7 in fbPolyline32 (pDrawable=0x8464d60, pGC=0x8408c38, mode=0, npt=4, ptsOrig=0xa2ec57d4)
    at ../../fb/fbbits.h:748
        _pPix = <value optimized out>
        pts = (INT32 *) 0xa2ec57dc
        xoff = 141
        yoff = -32741
        bias = 216
        pBox = <value optimized out>
        dstStride = 2048
        dstXoff = <value optimized out>
        dstYoff = <value optimized out>
        bits = (CARD32 *) 0x87a51244
        bitsBase = (CARD32 *) 0x97a0b234
        xor = 0
        and = 0
        dashoffset = 0
        ul = -2146893824
        lr = -2101608305
        pt1 = <value optimized out>
        pt2 = <value optimized out>
        e = -8
        e1 = 0
        e3 = -16
        len = <value optimized out>
        stepmajor = 1
        stepminor = <value optimized out>
        octant = <value optimized out>
#1  0xb7acd61c in fbPolyLine (pDrawable=0x8464d60, pGC=0xfffffff8, mode=0, npt=4, ppt=0xa2ec57d4)
    at ../../fb/fbline.c:142
        line = (void (*)(DrawablePtr, GCPtr, int, int, DDXPointPtr)) 0
#2  0xb7a6b592 in XAAPolylinesFallback (pDraw=0x8464d60, pGC=0x8408c38, mode=0, npt=4, pptInit=0xa2ec57d4)
    at ../../../../hw/xfree86/xaa/xaaFallback.c:139
        infoRec = (XAAInfoRecPtr) 0x8222598
        oldFuncs = (GCFuncs *) 0x81ef380
#3  0x0817686b in damagePolylines (pDrawable=0x8464d60, pGC=0x8408c38, mode=0, npt=4, ppt=0xa2ec57d4)
    at ../../../miext/damage/damage.c:993
        pGCPriv = (DamageGCPrivPtr) 0x8408ce0
        oldFuncs = (GCFuncs *) 0x81efb80
#4  0x0808bd40 in ProcPolyLine (client=0x8416808) at ../../dix/dispatch.c:1847
        npoint = 0
        pGC = (GC *) 0x8408c38
        pDraw = (DrawablePtr) 0x8464d60
#5  0x08154a11 in XaceCatchDispatchProc (client=0x8416808) at ../../Xext/xace.c:281
        major = 65
#6  0x0808ed3f in Dispatch () at ../../dix/dispatch.c:457
        result = <value optimized out>
        client = (ClientPtr) 0x8416808
        nready = 0
        start_tick = 5600
#7  0x08076e85 in main (argc=5, argv=0xbfffc114, envp=Cannot access memory at address 0x8
) at ../../dix/main.c:477
        pScreen = <value optimized out>
        i = <value optimized out>
        error = -1073757908
        xauthfile = <value optimized out>
        alwaysCheckForInput = {0, 1}
Comment 1 Erik 2008-02-28 04:34:39 UTC
The link here to the PDF file seems to point at an xorg.conf file. The debian bug's link to the PDF file is broken. So the bug can not be reproduced.
Comment 2 Brice Goglin 2008-02-28 05:12:33 UTC
The link to the PDF file was wrong, it should be
http://bugs.debian.org/cgi-bin/bugreport.cgi?msg=54;filename=chapter64.pdf;att=1;bug=320627

I can't attach the file here because it is too big (15MB).

I just downloaded the file (a bit slow) and open it with xpdf on Xserver 1.4.0.90 and pixman 0.9.6, it crashed pretty quickly after I played with the scrollbar on the left.

So yes, this bug can still be reproduced very easily.
Comment 3 Erik 2008-02-28 05:28:05 UTC
Crashed my X server too (Gentoo package x11-base/xorg-server-1.3.0.0-r5).
Comment 4 Julien Cristau 2008-02-28 07:38:43 UTC
On Thu, Feb 28, 2008 at 05:28:05 -0800, bugzilla-daemon@freedesktop.org wrote:

> --- Comment #3 from Erik <esigra@gmail.com>  2008-02-28 05:28:05 PST ---
> Crashed my X server too (Gentoo package x11-base/xorg-server-1.3.0.0-r5).

Crashes Xephyr too (from server-1.4-branch, as input in master is broken atm):
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb7bea6b0 (LWP 24322)]
0x080a2c15 in fbPolyline32 (pDrawable=0x8526ae0, pGC=0x84b19e8, mode=0, npt=1, 
    ptsOrig=0xb7383698) at ../../fb/fbbits.h:752
752                             STORE(bits,xor);
(gdb) p *bits
Cannot access memory at address 0xb7979c5c
(gdb) info locals
_pPix = <value optimized out>
pts = (INT32 *) 0xb73836a4
xoff = 9
yoff = -32554
bias = 216
dstStride = 640
dstXoff = <value optimized out>
dstYoff = <value optimized out>
bits = (CARD32 *) 0xb7979c5c
bitsBase = (CARD32 *) 0xb2b2cc2c
xor = 0
and = 0
dashoffset = 0
ul = 2135228416
lr = -2135424881
pt1 = <value optimized out>
pt2 = -2147287032
e = -59973
e1 = 8
e3 = -131064
len = 64836
stepmajor = -640
stepminor = -1
octant = <value optimized out>
(gdb) p *pDrawable
$7 = {type = 0 '\0', class = 1 '\001', depth = 24 '\030', 
  bitsPerPixel = 32 ' ', id = 10485859, x = 9, y = -32554, width = 1216, 
  height = 47988, pScreen = 0x81c2a18, serialNumber = 4275}

The value of pDrawable->y seems broken.

Cheers,
Julien
Comment 5 Daniel Stone 2009-08-31 18:33:44 UTC
punting this one, too ...
Comment 6 Julien Cristau 2010-06-06 09:01:19 UTC
(In reply to comment #4)
> Crashes Xephyr too (from server-1.4-branch, as input in master is broken atm):

still reproducible with 1.7...
Comment 7 Adam Jackson 2010-08-17 12:27:58 UTC
Created attachment 37924 [details] [review]
0001-fb-Fix-overflow-computation-in-lines-arcs-polys-glyp.patch

This patch is better, but not right.  I can still get it to crash at the STORE(bits,xor) in the polyline template if I play with it long enough, although it seems to be a function of the size of the xpdf window (and thus, I suppose, on the scrolling granularity, since larger windows crash which would have finer granularity).

This also exposes an xpdf bug, which is that it tries to do guffaw scrolling far larger than X says you can, but that's just a client bug like any other.
Comment 8 Adam Jackson 2010-08-19 13:21:54 UTC
Created attachment 37994 [details] [review]
11503.patch

This one works though.
Comment 9 Keith Packard 2010-08-19 19:11:13 UTC
Ok, I think I understand the issue now (too many numbers). The target window is taller than 32768 pixels, and yet the origin is way above the screen. So, the valid range of coordinates within the window is actually across the 32768 boundary, with some larger than int16 can hold.

The fb code cannot ever draw to coordinates above 32767, even with relative coordinates as those are converted to absolute coordinates before being drawn.

The example provided demonstrates how this breaks down -- a negative 'y' value is within the clip bounds when all are interpreted as int16 values.

It's not the fancy clipping code at fault, just the limited range of the coordinate system.

It should suffice to check for negative incoming coordinates (as those are never valid in any drawable), which is the piece Adam added in his working patch. Alternatively, the clip box could be restricted to values below 32768. Checking for negative coordinates is a whole lot simpler.

This patch should fix the problem, by looking for negative coordinates in the incoming coordinate:

-#define isClipped(c,ul,lr)  ((((c) - (ul)) | ((lr) - (c))) & 0x80008000)
+#define isClipped(c,ul,lr)  (((c) | ((c) - (ul)) | ((lr) - (c))) & 0x80008000)
Comment 10 Keith Packard 2010-08-20 09:58:04 UTC
Created attachment 38018 [details]
Creates window with visible coordinates > 32767 and attempts to draw

Here's a short program which exhibits this bug, crashing the X server without a lot of fussing around. My trivial (reject negative coords) patch resolves this issue.
Comment 11 Keith Packard 2010-08-20 10:14:24 UTC
Ajax reviewed my short patch and it has been merged to master in 3e56efcfb63677cd8574e1e435e61d96f79ea536


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.