Bug 12398 - Random crasher in libpixman
Summary: Random crasher in libpixman
Status: RESOLVED FIXED
Alias: None
Product: pixman
Classification: Unclassified
Component: pixman (show other bugs)
Version: 0.1.3
Hardware: x86-64 (AMD64) All
: medium major
Assignee: Søren Sandmann Pedersen
QA Contact: Søren Sandmann Pedersen
URL:
Whiteboard:
Keywords:
: 12783 (view as bug list)
Depends on:
Blocks:
 
Reported: 2007-09-11 22:20 UTC by Priit Laes (irc: plaes)
Modified: 2007-11-05 12:55 UTC (History)
9 users (show)

See Also:
i915 platform:
i915 features:


Attachments
gdb core dump session with gcc4.2 -g2 libpixman (13.36 KB, text/plain)
2007-09-17 18:11 UTC, Jan Christoph Nordholz
Details
Patch to fix 64-bit arithmetic flaw (831 bytes, patch)
2007-10-04 00:06 UTC, Francis Whittle
Details | Splinter Review

Description Priit Laes (irc: plaes) 2007-09-11 22:20:22 UTC
I got following random crasher in pixman library...

Backtrace:
0: /usr/bin/X(xf86SigHandler+0x6a) [0x4748d2]
1: /lib/libc.so.6 [0x2b9c71676d00]
2: /usr/lib/libpixman-1.so.0 [0x2b9c70d5063d]
3: /usr/lib/libpixman-1.so.0 [0x2b9c70d5645e]
4: /usr/lib/libpixman-1.so.0(pixman_composite_rect_general+0x330) [0x2b9c70d54196]
5: /usr/lib/libpixman-1.so.0 [0x2b9c70d5c558]
6: /usr/lib/libpixman-1.so.0(pixman_image_composite+0x70d) [0x2b9c70d5b9af]
7: /usr/lib64/xorg/modules//libfb.so(fbComposite+0x19c) [0x2b9c73d5941f]
8: /usr/lib64/xorg/modules//libxaa.so(XAAComposite+0x21b) [0x2b9c73f9d5ec]
9: /usr/lib64/xorg/modules//libxaa.so [0x2b9c73fb7aad]
10: /usr/lib64/xorg/modules/drivers//i810_drv.so(i830_xaa_composite+0x172) [0x2b9c72fdcf9d]
11: /usr/bin/X [0x519d9d]
12: /usr/bin/X [0x508c99]
13: /usr/bin/X(Dispatch+0x2f5) [0x44d60f]
14: /usr/bin/X(main+0x47f) [0x437135]
15: /lib/libc.so.6(__libc_start_main+0xe3) [0x2b9c71664553]
16: /usr/bin/X(FontFileCompleteXLFD+0x269) [0x4364f9]

Fatal server error:
Caught signal 11.  Server aborting
Comment 1 Søren Sandmann Pedersen 2007-09-13 14:01:03 UTC
Is this reproducable?

If so, can you get a stacktrace with debug symbols? Depending on your distribution you will need to get debug packages, then ssh in from another machine and attach gdb to the X server.

The most likely cause of this is pixman getting passed an invalid pointer, so it may be more productive to file this bug against the intel driver.

Comment 2 Priit Laes (irc: plaes) 2007-09-17 02:21:19 UTC
#0  0x00002b20b26f563d in fbFetchPixel_x8r8g8b8 (image=0x113e110, bits=0x400dab4a0, offset=0, indexed=0x0)
    at pixman-compose.c:710
No locals.
#1  0x00002b20b26fb4b3 in fbFetchTransformed (pict=0x113e110, x=<value optimized out>, y=<value optimized out>,
    width=<value optimized out>, buffer=0x7ffff8b6c560, mask=0x7ffff8b6c570, maskBits=4278190080)
    at pixman-compose.c:3892
        y1 = -1
        tl = <value optimized out>
        br = <value optimized out>
        x1_out = 0
        y2_out = 0
        x1 = 0
        y2 = <value optimized out>
        distx = 0
        idistx = 256
        b = (uint32_t *) 0x400dab4a0
        r = <value optimized out>
        x2_out = 0
        x2 = 1
        disty = 51
        tr = <value optimized out>
        bl = <value optimized out>
        y1_out = <value optimized out>
        bits = (uint32_t *) 0xdab4a0
        stride = <value optimized out>
        fetch = (fetchPixelProc) 0x2b20b26f563a <fbFetchPixel_x8r8g8b8>
        v = {vector = {6, -52438, 65536}}
        i = <value optimized out>
        box = {x1 = 0, y1 = 0, x2 = 2, y2 = 32}
        indexed = (const pixman_indexed_t *) 0x0
        affine = 1
#2  0x00002b20b26f9196 in pixman_composite_rect_general (data=0x7ffff8b72560, scanline_buffer=0x0)
    at pixman-compose.c:4394
No locals.
#3  0x00002b20b2701558 in pixman_image_composite_rect (op=PIXMAN_OP_OVER, src=0x113e110, mask=0xa90060,
    dest=0x151bc00, src_x=385, src_y=337, mask_x=0, mask_y=0, dest_x=685, dest_y=5120, width=2, height=33)
    at pixman-pict.c:1381
        compose_data = {op = 3 '\003', src = 0x113e110, mask = 0xa90060, dest = 0x151bc00, xSrc = 385, ySrc = 337,
  xMask = 0, yMask = 0, xDest = 685, yDest = 5120, width = 2, height = 33}
        _scanline_buffer = {16777215, 16777215, 16777215, 16777215, 855638016, 855638016,
  16777215 <repeats 5618 times>, 8415552, 0, 4890944, 0, 16777215, 29622272, 39714816, 606, 16777215, 16777215,
  16777215, 16777215, 16777215, 16777215, 8601872, 0, 612, 0, 456, 0, 8861408, 0, 4172750504, 32767, 8895600, 0,
  5081084, 0, 8601872, 0, 4172750508, 32767, 2, 0, 5081323, 0, 500175090, 0, 8861408, 0, 2, 0, 4569214, 0, 4, 0, 4,
  0, 8861408, 0, 2, 0, 6, 0, 4572416, 0, 16777215 <repeats 20 times>, 4172750508, 32767, 16777215, 0, 2, 3,
  500175090, 1, 8850784, 0, 16777215, 0, 16777215, 500175090, 456, 612, 16777215, 16777215, 10, 0, 9744656, 0, 0,
  0, 8079776, 0, 8861408, 0, 1, 0, 4761198, 0, 2, 16777215, 9744656, 0, 2, 0, 2, 0, 0, 0, 0, 0, 8861408, 0, 1, 0,
  4294967295, 0, 4761500, 0, 48, 16777215, 4172750912, 32767, 4172750688, 32767, 16777215, 16777215, 16777215,
  16777215, 16777215, 16777215, 16777215, 16777215, 16777215, 16777215, 16, 48, 4172750944, 32767, 4172750752,
  32767, 16777215, 16777215, 16777215, 16777215, 16777215, 16777215, 0, 0, 3003791427, 11040, 4172751072, 32767,
  496, 0, 16777215 <repeats 18 times>, 0, 0, 0, 0, 0, 0, 8822848, 0, 3032123399, 11040, 16777215, 16777215,
  8822448, 0, 16777215, 16777215, 4654941, 0 <repeats 37 times>...}
        scanline_buffer = (uint32_t *) 0x7ffff8b6c560
#4  0x00002b20b27009af in pixman_image_composite (op=PIXMAN_OP_OVER, pSrc=0x113e110, pMask=0xa90060,
    pDst=0x151bc00, xSrc=385, ySrc=337, xMask=0, yMask=0, xDst=685, yDst=5120, width=<value optimized out>,
    height=<value optimized out>) at pixman-pict.c:1287
        srcRepeat = 0
        maskRepeat = 0
        srcTransform = <value optimized out>
        maskTransform = <value optimized out>
        maskAlphaMap = <value optimized out>
        func = (CompositeFunc) 0x2b20b2701429 <pixman_image_composite_rect>
        mmx_setup = 1
#5  0x00002b20b56fe41f in fbComposite (op=16 '\020', pSrc=0x9de4f0, pMask=0xb85970, pDst=0x20c9ce0,
    xSrc=<value optimized out>, ySrc=<value optimized out>, xMask=0, yMask=<value optimized out>, xDst=385,
    yDst=337, width=2, height=33) at fbpict.c:185
        src = (pixman_image_t *) 0x113e110
        mask = (pixman_image_t *) 0xa90060
        dest = (pixman_image_t *) 0x151bc00
#6  0x00002b20b59425ec in XAAComposite (op=3 '\003', pSrc=0x9de4f0, pMask=0xb85970, pDst=0x20c9ce0, xSrc=385,
    ySrc=<value optimized out>, xMask=0, yMask=0, xDst=385, yDst=337, width=2, height=33) at xaaPict.c:545
        pScreen = (ScreenPtr) 0x834110
        infoRec = (XAAInfoRecPtr) 0x839c90
#7  0x00002b20b595caad in cwComposite (op=16 '\020', pSrcPicture=<value optimized out>, pMskPicture=0xb85970,
    pDstPicture=0x20c9ce0, xSrc=385, ySrc=337, xMsk=0, yMsk=0, xDst=385, yDst=337, width=<value optimized out>,
    height=<value optimized out>) at cw_render.c:274
        ps = (PictureScreenPtr) 0x837810
        pCwScreen = (cwScreenPtr) 0x84e730
        src_picture_x_off = 0
        src_picture_y_off = 0
        pBackingSrcPicture = (PicturePtr) 0x9de4f0
        msk_picture_x_off = 0
        msk_picture_y_off = 0
        pBackingMskPicture = (PicturePtr) 0xb85970
        dst_picture_x_off = 0
        dst_picture_y_off = 0
        pBackingDstPicture = (PicturePtr) 0x0
#8  0x00002b20b4981f9d in i830_xaa_composite (op=16 '\020', pSrc=0x9de4f0, pMask=0xb85970, pDst=0x20c9ce0,
    xSrc=385, ySrc=337, xMask=0, yMask=<value optimized out>, xDst=385, yDst=337, width=2, height=33)
    at i830_xaa.c:865
        pScreen = (ScreenPtr) 0x834110
        pScrn = (ScrnInfoPtr) 0x802f30
        pI830 = (I830Ptr) 0x8058f0
        ps = (PictureScreenPtr) 0x837810
        pSrcPixmap = (PixmapPtr) 0x50176f
        pDstPixmap = (PixmapPtr) 0x21000200000000
        region = {extents = {x1 = 22896, y1 = 184, x2 = 0, y2 = 0}, data = 0x0}
        pbox = <value optimized out>
        nbox = <value optimized out>
        i = <value optimized out>
#9  0x0000000000519d9d in damageComposite (op=16 '\020', pSrc=0x9de4f0, pMask=0xb85970, pDst=0x20c9ce0, xSrc=-1,
    ySrc=-32768, xMask=0, yMask=<value optimized out>, xDst=<value optimized out>, yDst=<value optimized out>,
    width=<value optimized out>, height=<value optimized out>) at damage.c:580
        ps = (PictureScreenPtr) 0x837810
        pScrPriv = (DamageScrPrivPtr) 0x84d990
#10 0x0000000000502e52 in miTrapezoids (op=16 '\020', pSrc=0x9de4f0, pDst=0x20c9ce0, maskFormat=0x838cf8, xSrc=385,
    ySrc=337, ntrap=0, traps=0x14b5a88) at mitrap.c:174
        pPicture = (PicturePtr) 0xb85970
        bounds = {x1 = 385, y1 = 337, x2 = 387, y2 = 370}
        pScreen = <value optimized out>
        ps = (PictureScreenPtr) 0x837810
#11 0x00002b20b595cdc8 in cwTrapezoids (op=16 '\020', pSrcPicture=<value optimized out>, pDstPicture=0x20c9ce0,
    maskFormat=0x838cf8, xSrc=385, ySrc=337, ntrap=1, traps=0x14b5a60) at cw_render.c:364
        ps = (PictureScreenPtr) 0x837810
        pCwScreen = (cwScreenPtr) 0x84e730
        src_picture_x_off = 0
        src_picture_y_off = 0
        pBackingSrcPicture = (PicturePtr) 0x9de4f0
        dst_picture_x_off = 0
        dst_picture_y_off = 0
        pBackingDstPicture = (PicturePtr) 0x0
        i = <value optimized out>
#12 0x000000000050b14f in ProcRenderTrapezoids (client=0xd7cbb0) at render.c:820
        pSrc = (PicturePtr) 0x400dab4a0
        pDst = (PicturePtr) 0x0
        pFormat = (PictFormatPtr) 0x0
#13 0x000000000044d60f in Dispatch () at dispatch.c:502
        clientReady = <value optimized out>
        result = <value optimized out>
        client = (ClientPtr) 0xd7cbb0
        nready = 0
        start_tick = 22471840
#14 0x0000000000437135 in main (argc=9, argv=0x7ffff8b73188, envp=<value optimized out>) at main.c:452
        pScreen = <value optimized out>
        i = 1
        error = 0
        xauthfile = <value optimized out>
        alwaysCheckForInput = {0, 1}
Comment 3 Brice Goglin 2007-09-17 09:44:47 UTC
The same problem has been reported by Jan Christoph Nordholz with a similar debugging backtrace at
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442829

He says:
"the current experimental version of the X server dies when I start a "more complex" client like xdm or enlightenment17 (I guess it's related to a specific feature the client requests - a simple xterm doesn't cause the crash)."

Elimar Riesebieter also reported the same problem at
  http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442852

They both seem to be using i686, not amd64 as Priit Laes.

Comment 4 Jan Christoph Nordholz 2007-09-17 18:09:49 UTC
Hi,

compiling libpixman with gcc4.1 solves the problem without further changes.
To make the debugging easier, I'm attaching a more verbose gdb session with a 'gcc-4.2 -g2'-compiled libpixman.
Comment 5 Jan Christoph Nordholz 2007-09-17 18:11:22 UTC
Created attachment 11610 [details]
gdb core dump session with gcc4.2 -g2 libpixman
Comment 6 Eugene St Leger 2007-09-20 05:23:17 UTC
Further to comment #3 in this bug, linked here:
https://bugs.freedesktop.org/show_bug.cgi?id=12398#c3

The xserver crashed for me in exactly the same manner as the debian bug linked in comment #3 (signal 4 AKA SIGILL - an illegal instruction encountered).  I link to debian's bug report again:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442829

I run Gentoo.  Its pixman package (x11-libs/pixman-0.9.5) is a new requirement of the latest xserver package (x11-base/xorg-server-1.4-r1).  pixman compiles with both MMX and SSE instructions (-mmmx and -msse), even though I don't have -msse in my CFLAGS, and my processor (AMD Duron) doesn't have SSE support, perfectly explaining my signal 4 crash.  I managed to compile without -msse, and I don't experience the absolutely reproducible crashes I was getting before.  I've filed a Gentoo bug here:
http://bugs.gentoo.org/show_bug.cgi?id=193138

Interestingly, lots of recent commit summaries at the pixman git repository
involve the MMX support, but the posts there are a bit cryptic for me:
http://gitweb.freedesktop.org/?p=pixman.git;a=shortlog

Also, comment #4 in the following pixman bug about MMX and SSE support looks very relevant:
https://bugs.freedesktop.org/show_bug.cgi?id=4706#c4

In the aforementioned debian bug report, a fixed pixman package has been announced, but I haven't examined it yet:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442829#59
Comment 7 Alan Hourihane 2007-09-20 05:42:04 UTC
Right. I spoke to Soren on IRC about this yesterday.

The best course of action for now is to remove -msse from pixman.
Comment 8 Søren Sandmann Pedersen 2007-09-20 10:11:12 UTC
Note: this bug is about a SIGSEGV, not SIGILL. This is not caused by -msse, but most likely by pixman getting passed an invalid pointer.

Please file the SIGILL issue with gcc 4.2 as a separate bug.
Comment 9 Jan de Groot 2007-10-03 09:44:26 UTC
On Archlinux we have the same bug, reported on amd64. This bug appears with the binary nvidia driver, but also with the opensource vesa driver.
Evince 2.20 is one of the apps that can trigger this bug, see the archlinux report for more info:
http://bugs.archlinux.org/task/8188
Comment 10 Francis Whittle 2007-10-04 00:06:12 UTC
Created attachment 11896 [details] [review]
Patch to fix 64-bit arithmetic flaw

100% reproducable when launching kiba-dock on AMD64 System.
Appears to be being caused by this little gem in pixman-compose.c (at least in my case):

b = bits + (y1)*stride;

This works okay until y1 is negative (and stride is non-zero) - y1 is implicitly cast to an unsigned integer (for some reason 32-bit, is this because y1 is 32-bit?) before processing the statement.  This is fine on 32-bit platforms, but on 64-bit has the undesirable result of producing a large positive integer (> 2^32) which is an inaccessible memory address, therefore try to read it (as it done shortly afterwards) results in a segfault as observed.

The attached patch (explicit cast to signed integer) does the trick for me, there may be other points where this indiscretion is present.
Comment 11 Andrea Cimitan 2007-10-08 03:10:21 UTC
I have that crash in Archlinux with i686 (so it's not just 64bit).
http://bugs.archlinux.org/task/8252#comment19591
Comment 12 Francis Whittle 2007-10-08 04:31:57 UTC
(In reply to comment #11)
Andrea, the trace in the Archlinux bug report suggests that the segmentation fault is occuring in a different location -- actually inside pixman_image_composite as opposed to one of the fbFetchPixel_ functions where Priit's setup is crashing.
Comment 13 Michel Dänzer 2007-10-12 00:58:39 UTC
*** Bug 12783 has been marked as a duplicate of this bug. ***
Comment 14 Michel Dänzer 2007-10-12 00:59:50 UTC
Søren, any reason against applying this patch?
Comment 15 Priit Laes (irc: plaes) 2007-10-15 05:54:47 UTC
(In reply to comment #10)
> Created an attachment (id=11896) [details]
> Patch to fix 64-bit arithmetic flaw
> 
> The attached patch (explicit cast to signed integer) does the trick for me,
> there may be other points where this indiscretion is present.
> 

Works for me too...
Comment 16 Søren Sandmann Pedersen 2007-10-16 10:16:58 UTC
Michel, no, feel free to apply it.

Comment 17 Søren Sandmann Pedersen 2007-10-17 16:24:10 UTC
I have committed this locally; I'll push it out and later today. I'd appreciate any testing, on both 64 and 32 bit.

After testing, I'll make a new release.


commit ab6743b17074dfedffb0ee32fe2e37cad03769bf
Author: Søren Sandmann <sandmann@redhat.com>
Date:   Wed Oct 17 19:20:55 2007 -0400

    Make stride signed in various places. This should fix some the
    crashers that people have reported in bug 12398.

Comment 18 Andrea Cimitan 2007-10-17 16:27:44 UTC
Could you fix http://bugs.archlinux.org/task/8294
before release??
It's more annoying... As I can't start X server...
Comment 19 Søren Sandmann Pedersen 2007-10-17 17:07:47 UTC
I have pushed the fix now. Testing of git HEAD or this tarball:

    http://www.freedesktop.org/~sandmann/pixman-0.9.6-testing.tar.gz

would be much appreciated.

Comment 20 Søren Sandmann Pedersen 2007-10-17 17:59:40 UTC
(In reply to comment #18)
> Could you fix http://bugs.archlinux.org/task/8294
> before release??
> It's more annoying... As I can't start X server...
> 

You need to either reproduce with the open source nv driver, or somehow show that the bug is likely to be in pixman. Then get a backtrace with debug symbols and file a separate bug (here, not in archlinux's bugzilla).
Comment 21 Lars Heidieker 2007-10-18 01:59:31 UTC
(In reply to comment #17)
> I have committed this locally; I'll push it out and later today. I'd appreciate
> any testing, on both 64 and 32 bit.
> 
> After testing, I'll make a new release.
> 
> 
> commit ab6743b17074dfedffb0ee32fe2e37cad03769bf
> Author: Søren Sandmann <sandmann@redhat.com>
> Date:   Wed Oct 17 19:20:55 2007 -0400
> 
>     Make stride signed in various places. This should fix some the
>     crashers that people have reported in bug 12398.
> 

I saw the originla backtrace on my gentoo amd64 box with th crash in pixman,
the signed fix was applied (it is in gentoo in the unstable arch) fixed it.
I run a gnome desktop and the crash appeared with evience I had no other crashes.
But viewing a pdf did crash is every time I resized a window horizontaly.
That's fixed I did not hav a single crash now.
(I don't think it has to do with the driver but mine is the open source ati with dri support X550 Card)
Comment 22 Francis Whittle 2007-10-18 18:45:20 UTC
(In reply to comment #19)
> I have pushed the fix now. Testing of git HEAD or this tarball:
> 
>     http://www.freedesktop.org/~sandmann/pixman-0.9.6-testing.tar.gz
> 
> would be much appreciated.
> 

Tarball works perfectly for me.
Comment 23 Felix Apitzsch 2007-11-01 14:42:39 UTC
(In reply to comment #19)
> I have pushed the fix now. Testing of git HEAD or this tarball:
> 
>     http://www.freedesktop.org/~sandmann/pixman-0.9.6-testing.tar.gz
> 
> would be much appreciated.
> 

With pixman-0.9.6 my system still crashes (exatly the same way to reproduce), but now the backtrace does not point to pixman anymore:

Backtrace:
0: /usr/bin/X(xf86SigHandler+0x6a) [0x487f1b]
1: /lib/libc.so.6 [0x2b0be5b42730]
2: /usr/lib64/xorg/modules/extensions//libGLcore.so(_mesa_update_state_locked+0x872) [0x2b0bf097c812]
3: /usr/lib64/xorg/modules/extensions//libGLcore.so(_mesa_update_state+0x11) [0x2b0bf097c9cc]
4: /usr/lib64/xorg/modules/extensions//libGLcore.so(_mesa_GetIntegerv+0x245) [0x2b0bf09421c5]
5: /usr/lib64/xorg/modules/extensions//libglx.so [0x2b0be6e97e60]
6: /usr/lib64/xorg/modules/extensions//libglx.so [0x2b0be6e90333]
7: /usr/bin/X(Dispatch+0x2ec) [0x44f92e]
8: /usr/bin/X(main+0x479) [0x437106]
9: /lib/libc.so.6(__libc_start_main+0xf4) [0x2b0be5b2eb74]
10: /usr/bin/X(FontFileCompleteXLFD+0x259) [0x4364d9]

Fatal server error:
Caught signal 11.  Server aborting

I am starting to wander, if the original bug #12783 really is a duplicte of this one.
Comment 24 Søren Sandmann Pedersen 2007-11-01 17:02:39 UTC
Just because the reproduction method is the same does not mean it's the same bug. This is a different stacktrace; we should track it in a different bug.

Please file a new bug against the X server (the GL component). If possible, plese get a stacktrace with debug symbols for libglx and libGL.

Thanks

(I am closing this bug - I believe it is fixed with 0.9.6 and that you are seeing something different).
Comment 25 Coleman Kane 2007-11-05 12:12:06 UTC
I ran into this crasher in pixman 0.9.5 under FreeBSD/amd64. I have since upgraded to pixman 0.9.6 and it seems to fix this bug. I just wanted to report success. Will be opening FreeBSD PR to update the version in ports.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.