Bug 39193 - [llvmpipe and r600g] glCheckFramebufferStatusEXT segfaults in Gallium when checking status on a framebuffer bound to a texture that's bound to a pixmap
[llvmpipe and r600g] glCheckFramebufferStatusEXT segfaults in Gallium when ch...
Product: Mesa
Classification: Unclassified
Component: Drivers/Gallium/r600
x86 (IA32) Linux (All)
: medium normal
Assigned To: Default DRI bug account
Depends on:
  Show dependency treegraph
Reported: 2011-07-13 07:41 UTC by Simon Farnsworth
Modified: 2011-09-22 08:07 UTC (History)
0 users

See Also:
i915 platform:
i915 features:

Test program demonstrating the crash (13.21 KB, text/x-csrc)
2011-07-13 07:41 UTC, Simon Farnsworth
st/mesa: Finalize texture on render-to-texture. (1.56 KB, patch)
2011-09-14 05:06 UTC, Michel Dänzer
Details | Splinter Review
Expected output of the test program (517.20 KB, image/png)
2011-09-14 08:34 UTC, Simon Farnsworth

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Farnsworth 2011-07-13 07:41:28 UTC
Created attachment 49039 [details]
Test program demonstrating the crash

In our OpenGL compositor, we double-buffer some windows by using GL_EXT_framebuffer_object and glTexCopyImage to copy the window backing pixmap to a new texture.

This crashes on r600g (but not Intel DRI) drivers, in Gallium's implementation of glCheckFramebufferStatusEXT due to a NULL deference.

I've attached a program that demonstrates the crash, using a scratch pixmap created by the program instead of a window backing store pixmap created by XComposite. It's built with:

gcc -O2 -ggdb -Wall -Werror -o framebuffertexturetest framebuffertexturetest.c -lX11 -lGL -lGLEW -lGLU

When I run this against git versions of Mesa, I get the following backtrace (via gdb):

(gdb) t a a bt full

Thread 1 (Thread 0xb7a1e6d0 (LWP 971)):
#0  0xb771a9b4 in st_validate_attachment (ctx=0x8162248, screen=0x8063968, att=0x81aabd8, bindings=2) at state_tracker/st_cb_fbo.c:472
        stObj = 0x81aa4b8
        format = <optimized out>
        texFormat = <optimized out>
#1  0xb771ab6a in st_validate_framebuffer (ctx=0x8162248, fb=0x81aa9f8) at state_tracker/st_cb_fbo.c:570
        att = 0x81aabd8
        format = <optimized out>
        screen = 0x8063968
        depth = 0x81aab68
        stencil = 0x81aab84
        i = <optimized out>
        first_format = <optimized out>
        mixed_formats = 1 '\001'
#2  0xb76be381 in _mesa_test_framebuffer_completeness (ctx=0x8162248, fb=0x81aa9f8) at main/fbobject.c:866
        numImages = <optimized out>
        intFormat = <optimized out>
        minWidth = <optimized out>
        minHeight = <optimized out>
        maxWidth = <optimized out>
        maxHeight = <optimized out>
        numSamples = <optimized out>
        i = <optimized out>
        j = <optimized out>
        __PRETTY_FUNCTION__ = "_mesa_test_framebuffer_completeness"
#3  0xb76bff48 in _mesa_CheckFramebufferStatusEXT (target=36160) at main/fbobject.c:1850
        buffer = 0x81aa9f8
        ctx = 0x8162248
#4  0xb7e6d6e8 in glCheckFramebufferStatusEXT (target=36160) at ../../../src/mapi/glapi/glapitemp.h:5728
No locals.
#5  0x08049a2f in render_to_texture (target_texture=1) at framebuffertexturetest.c:151
        framebuffer = 1
        value = <optimized out>
        pixmapAttribs = {8406, 8412, 8405, 8410, 0}
        glx_pixmap = 2097159
        gc = <optimized out>
        attrib = {x = 0, y = 0, width = 1920, height = 1200, border_width = 0, depth = 24, visual = 0x8056ab8, root = 343, class = 1, bit_gravity = 0, win_gravity = 1, backing_store = 0, backing_planes = 4294967295, backing_pixel = 0, 
          save_under = 0, colormap = 2097153, map_installed = 0, map_state = 2, all_event_masks = 0, your_event_mask = 0, do_not_propagate_mask = 0, override_redirect = 0, screen = 0x80566c8}
        visualid = 273
        fbconfigs = 0x81a74f0
        nfbconfigs = 144
        status = <optimized out>
        bound_texture = 4
        i = <optimized out>
        pixmap = 2097157
        visinfo = <optimized out>
#6  main (argc=1, argv=0xbffff6e4) at framebuffertexturetest.c:344
        last_frame_time = <optimized out>
        start_time = <optimized out>
        ust = 577789755707940852
        msc = <optimized out>
        draw_texture = 1

My X stack is using the following components from git; everything else is from Fedora 15, apart from the kernel, which is airlied's drm-fixes:

dri2proto: commit 7fd18b15646a62bd82a4eb0eca60a34c1731813d
drm: commit 8d055890d90c3d92647e3d8b98d32630ef87c2c8
glproto: commit 6d7e398150dde1c15b7f62f485cb33bad8f4d5c6
libpciaccess: commit f9159b97834ba4b4e42a07953a33866e7ac90dbd
mesa: commit 5fe5d236c26b3b2428bc7395304e40cf21d3d3e1
pixman: commit e814b50877bf313619fbf777dcab98d39874f8a4
randrproto: commit b5c977ad70b845c278ef52c004d9e5fc3ec97cbf
xf86-input-evdev: commit 5069e053354ad6adfcbb08c962bff93b607dec3d
xf86-video-ati: commit e8d0d437957b15252dfad775796a3949ed50dbcf
xorg-font-util: commit 9f807dfb4e7bea7128b7538e41de78bb5994ca80
xorg-macros: commit 08dd5d30c137abe43d5b1673edc39eb569881d66
xproto: commit 06ebd5b88bee8c54222bde3178dc773776036802
xserver: commit 73c05be80a1d682ff623e66f47e87403d94caef5

I'm going to start digging, see if I can work out where my texture goes missing, but all advice would be gratefully received; my goal is to keep the pixmap GPU-side if possible, while copying its contents to a separate texture.
Comment 1 Simon Farnsworth 2011-07-14 09:54:34 UTC
I'm going to hunt for a different way to do this; I can see what's going on, but I have no idea why.

glXBindTexImageEXT ends up calling into st_context_teximage (st_manager.c:523 onwards in my git checkout). When st_context_teximage is about to exit (returning TRUE), I can see that stObj and stImage have been successfully initialised, but that stImage->pt is NULL.

When I call glFramebufferTexture2DEXT, the same stObj as st_context_teximage set up is used; this step works, because it never looks at the pt member.

When I call glCheckFramebufferStatusEXT, I get burnt - st_validate_attachment deferences pt without checking it for NULL, and thus I segfault out.

I can't work out how stImage->pt is meant to be set; it looks like it might be a side-effect of pipe_resource_reference, but it isn't happening.
Comment 2 Simon Farnsworth 2011-09-12 15:16:53 UTC
Tom Stellard was kind enough to try this on his laptop for me at XDC - it fails on llvmpipe as well as r600g, suggesting it's a core Gallium bug.
Comment 3 Michel Dänzer 2011-09-14 05:06:23 UTC
Created attachment 51195 [details] [review]
st/mesa: Finalize texture on render-to-texture.

This patch fixes the crash, but I'm not sure it makes the test program work correctly.
Comment 4 Simon Farnsworth 2011-09-14 08:34:12 UTC
Created attachment 51200 [details]
Expected output of the test program

I won't be able to test this patch properly until I'm back in the office on Monday (my personal laptop doesn't have a working Mesa build setup on it) - in the meantime, I've run the test program on Intel, and taken a screenshot of the expected output (as it does look like random garbage).
Comment 5 Simon Farnsworth 2011-09-14 09:31:43 UTC
Tom's come to my rescue once again, and tested Michel's patch on his laptop - the output is exactly what I'd expect.

I will reopen this bug if it doesn't work correctly on the E-350 back at the office.
Comment 6 Michel Dänzer 2011-09-22 08:07:34 UTC
Note that a bug shouldn't be resolved before the fix lands in Git, but it has now.

BTW, I was confused for a while by a latent bug of the test program: It leaves bound_texture uninitialized, and when it ends up being 0, the texture fails to attach to the framebuffer, and that ends up incomplete.