Bug 12614

Summary:	segfault inside quadfunc_unfilled_rgba()
Product:	Mesa	Reporter:	Ademar Reis <ademar>
Component:	Other	Assignee:	mesa-dev
Status:	RESOLVED FIXED	QA Contact:
Severity:	normal
Priority:	medium	CC:	cmatsuoka
Version:	git
Hardware:	x86 (IA32)
OS:	Linux (All)
Whiteboard:
i915 platform:		i915 features:

Description Ademar Reis 2007-09-28 11:25:58 UTC

I'm geting segfaults with two different programs (metisse and neverball) when using drivers r200 and i915. The crashs happen inside quadfunc_unfilled_rgba().

I tried both mesa 7.0.1 and the version from git (origin/mesa_7_0_branch), both cause a segfault of the application. When running neverball, the xserver actually freezes untill neverball is killed (-9).

Below is a backtrace from neverball (a KDE game: http://icculus.org/neverball/).

I'll try to prepare a simpler testcase, but maybe the backtrace is of some help:

Starting program: /usr/games/neverball.bin
[Thread debugging using libthread_db enabled]
[New Thread -1218353456 (LWP 10800)]
[New Thread -1221067888 (LWP 10803)]
Mesa: CPU vendor: GenuineIntel
Mesa: CPU name: Intel(R) Celeron(R) M CPU        410  @ 1.46GHz
Mesa: MMX cpu detected.
Mesa: SSE cpu detected.
Mesa: Not testing OS support for SSE, leaving enabled.
Mesa: Mesa 7.0.1 DEBUG build Sep 28 2007 14:31:57
Mesa warning: couldn't open libtxc_dxtn.so, software DXTn compression/decompression unavailable

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1218353456 (LWP 10800)]
0xb6a64013 in quadfunc_unfilled_rgba (ctx=0x81d2140, v0=0, v1=1, v2=2, v3=3) at swrast_setup/ss_tritmp.h:201
201           GLubyte ef1 = VB->EdgeFlag[v1];
(gdb) bt
#0  0xb6a64013 in quadfunc_unfilled_rgba (ctx=0x81d2140, v0=0, v1=1, v2=2, v3=3) at swrast_setup/ss_tritmp.h:201
#1  0xb69de3e7 in _tnl_render_quads_verts (ctx=0x81d2140, start=0, count=4, flags=55) at tnl/t_vb_rendertmp.h:338
#2  0xb69dfc9f in run_render (ctx=0x81d2140, stage=0x820fccc) at tnl/t_vb_render.c:320
#3  0xb69d2984 in _tnl_run_pipeline (ctx=0x81d2140) at tnl/t_pipeline.c:158
#4  0xb691138e in intelRunPipeline (ctx=0x81d2140) at intel_tris.c:764
#5  0xb69d38b1 in _tnl_draw_prims (ctx=0x81d2140, arrays=0x81fdea8, prim=0x81fca04, nr_prims=1, ib=0x0, min_index=0,
    max_index=3) at tnl/t_draw.c:403
#6  0xb69c89e1 in vbo_exec_vtx_flush (exec=0x81fc8e0) at vbo/vbo_exec_draw.c:215
#7  0xb69c7493 in vbo_exec_FlushVertices (ctx=0x81d2140, flags=1) at vbo/vbo_exec_api.c:685
#8  0xb69623a0 in _mesa_PopMatrix () at main/matrix.c:274
#9  0x080507dc in sol_back (fp=0x80772c0, n=256, f=512, t=124.494003) at share/solid.c:214
#10 0x0805a1d4 in game_draw_back (pose=<value optimized out>, d=<value optimized out>, p=<value optimized out>)
    at ball/game.c:348
#11 0x0805ab46 in game_draw (pose=0, st=0) at ball/game.c:479
#12 0x0806486c in title_paint (id=19, st=0) at ball/st_title.c:108
#13 0x080580ff in st_paint () at share/state.c:69
#14 0x08065b2a in main (argc=Cannot access memory at address 0x1
) at ball/main.c:272
#15 0xb7bd1f90 in __libc_start_main () from /lib/i686/libc.so.6
#16 0x0804a641 in _start ()

Comment 1 Ademar Reis 2007-09-28 11:33:48 UTC

A different backtrace, now from fvwmComposer (part of the metisse project http://insitu.lri.fr/metisse/):

Program terminated with signal 11, Segmentation fault.
#0  0xb74b3013 in quadfunc_unfilled_rgba (ctx=0x80c5118, v0=0, v1=1, v2=2, v3=3) at swrast_setup/ss_tritmp.h:201
201           GLubyte ef1 = VB->EdgeFlag[v1];
(gdb) bt
#0  0xb74b3013 in quadfunc_unfilled_rgba (ctx=0x80c5118, v0=0, v1=1, v2=2, v3=3) at swrast_setup/ss_tritmp.h:201
#1  0xb742d3e7 in _tnl_render_quads_verts (ctx=0x80c5118, start=0, count=4, flags=55) at tnl/t_vb_rendertmp.h:338
#2  0xb742ec9f in run_render (ctx=0x80c5118, stage=0x8109f0c) at tnl/t_vb_render.c:320
#3  0xb7421984 in _tnl_run_pipeline (ctx=0x80c5118) at tnl/t_pipeline.c:158
#4  0xb736038e in intelRunPipeline (ctx=0x80c5118) at intel_tris.c:764
#5  0xb74228b1 in _tnl_draw_prims (ctx=0x80c5118, arrays=0x80f8108, prim=0x80f6c64, nr_prims=1, ib=0x0, min_index=0, max_index=3) at tnl/t_draw.c:403
#6  0xb74179e1 in vbo_exec_vtx_flush (exec=0x80f6b40) at vbo/vbo_exec_draw.c:215
#7  0xb7416493 in vbo_exec_FlushVertices (ctx=0x80c5118, flags=1) at vbo/vbo_exec_api.c:685
#8  0xb73cdad8 in _mesa_StencilFunc (func=514, ref=1, mask=1) at main/stencil.c:147
#9  0x08081035 in WindowRenderer::_shapeDisplay (this=0x83d1000, on=true) at WindowRenderer.cxx:1643
#10 0x0808b4f4 in WindowRenderer::display (this=0x83d1000, policy=nucleo::sgNode::NODL) at WindowRenderer.cxx:1676
#11 0xb7f2faf1 in nucleo::sgNode::displayGraph () from /usr/lib/libNucleo.so.0
#12 0xb7f2fe2f in nucleo::sgNode::display () from /usr/lib/libNucleo.so.0
#13 0xb7f2faf1 in nucleo::sgNode::displayGraph () from /usr/lib/libNucleo.so.0
#14 0x08061c0a in LayerManager::display (this=0x8399c48, policy=nucleo::sgNode::NODL) at LayerManager.cxx:114
#15 0xb7f2faf1 in nucleo::sgNode::displayGraph () from /usr/lib/libNucleo.so.0
#16 0xb7f30a2f in nucleo::sgViewpoint::display () from /usr/lib/libNucleo.so.0
#17 0xb7f2faf1 in nucleo::sgNode::displayGraph () from /usr/lib/libNucleo.so.0
#18 0x08059c8e in AScreen::draw (this=0x80b7fa0, force=false) at AScreen.cxx:3608
#19 0x08070327 in MetisseDesktop::handleRestackWindow (this=0x80b2cf0, id=20971836, nextId=6291464, transientFor=0, unmanagedFor=0, grabWin=0, duplicateFor=0, facadeReal=0, 
    flags=<value optimized out>) at MetisseDesktop.cxx:948
#20 0x0807ea6d in MetisseSource::_readMETISSEServer (this=0x80b38d0) at MetisseSource.cxx:1091
#21 0x0807ec46 in MetisseSource::react (this=0x80b38d0, obs=0x1) at MetisseSource.cxx:688
#22 0xb7ec95d2 in nucleo::ReactiveEngineImplementation::doReact () from /usr/lib/libNucleo.so.0
#23 0xb7ed483c in nucleo::sReactiveEngine::step () from /usr/lib/libNucleo.so.0
#24 0xb7ed42c0 in nucleo::sReactiveEngine::run () from /usr/lib/libNucleo.so.0
#25 0xb7ec9b5c in nucleo::ReactiveEngine::run () from /usr/lib/libNucleo.so.0
#26 0x0804eaf6 in main (argc=42, argv=0xbfd92f94) at compositor.cxx:227
#27 0xb75faf90 in __libc_start_main () from /lib/i686/libc.so.6
#28 0x0804e1e1 in _start ()

Comment 2 Ademar Reis 2007-09-28 14:34:56 UTC

Backtrace from FvwmCompositor when running on radeon:

#0  0xb748b56f in quadfunc_unfilled_rgba (ctx=0x80d25d8, v0=0, v1=1, v2=2, v3=3)
    at swrast_setup/ss_tritmp.h:201
#1  0xb7402e8b in _tnl_render_quads_verts (ctx=0x80d25d8, start=0, count=4, flags=55)
    at tnl/t_vb_rendertmp.h:338
#2  0xb7404743 in run_render (ctx=0x80d25d8, stage=0x812e7f4) at tnl/t_vb_render.c:320
#3  0xb73f7428 in _tnl_run_pipeline (ctx=0x80d25d8) at tnl/t_pipeline.c:158
#4  0xb7303955 in r200WrapRunPipeline (ctx=0x80d25d8) at r200_state.c:2590
#5  0xb73f8355 in _tnl_draw_prims (ctx=0x80d25d8, arrays=0x811ca00, prim=0x811b55c, nr_prims=1,
    ib=0x0, min_index=0, max_index=3) at tnl/t_draw.c:403
#6  0xb73ed495 in vbo_exec_vtx_flush (exec=0x811b438) at vbo/vbo_exec_draw.c:215
#7  0xb73ebf47 in vbo_exec_FlushVertices (ctx=0x80d25d8, flags=1) at vbo/vbo_exec_api.c:685
#8  0xb73a3624 in _mesa_StencilFunc (func=514, ref=1, mask=1) at main/stencil.c:147
#9  0x08081035 in WindowRenderer::_shapeDisplay (this=0x83a5c70, on=true)
    at WindowRenderer.cxx:1643
#10 0x0808b4f4 in WindowRenderer::display (this=0x83a5c70, policy=nucleo::sgNode::NODL)
    at WindowRenderer.cxx:1676
#11 0xb7f05af1 in nucleo::sgNode::displayGraph (this=0x83a5c70, policy=nucleo::sgNode::NODL)
    at sgNode.cxx:320
#12 0xb7f05e2f in nucleo::sgNode::display (this=0x83a59e0, policy=nucleo::sgNode::NODL)
    at sgNode.cxx:333
#13 0xb7f05af1 in nucleo::sgNode::displayGraph (this=0x83a59e0, policy=nucleo::sgNode::NODL)
    at sgNode.cxx:320
#14 0x08061c0a in LayerManager::display (this=0x8390fa0, policy=nucleo::sgNode::NODL)
    at LayerManager.cxx:114
#15 0xb7f05af1 in nucleo::sgNode::displayGraph (this=0x8390fa0, policy=nucleo::sgNode::NODL)
    at sgNode.cxx:320
#16 0xb7f06a2f in nucleo::sgViewpoint::display (this=0x8390de8, policy=nucleo::sgNode::NODL)
    at sgViewpoint.cxx:34
#17 0xb7f05af1 in nucleo::sgNode::displayGraph (this=0x8390de8, policy=nucleo::sgNode::NODL)
    at sgNode.cxx:320
#18 0x08059c8e in AScreen::draw (this=0x80b7fa0, force=false) at AScreen.cxx:3608
#19 0x0805b179 in AScreen::react (this=0x80b7fa0, obs=0x80b2cf0) at AScreen.cxx:3597
#20 0xb7e9f5d2 in nucleo::ReactiveEngineImplementation::doReact (this=0x80b7b08, obj=0x80b7fa0,
    obs=0x80b2cf0) at ReactiveEngine.cxx:52
#21 0xb7eaa83c in nucleo::sReactiveEngine::step (this=0x80b7b08, timeout=-1)
    at sReactiveEngine.cxx:144
#22 0xb7eaa2c0 in nucleo::sReactiveEngine::run (this=0x80b7b08) at sReactiveEngine.cxx:155
#23 0xb7e9fb5c in nucleo::ReactiveEngine::run () at ReactiveEngine.cxx:136
#24 0x0804eaf6 in main (argc=42, argv=0xbff47444) at compositor.cxx:227

Comment 3 Brian Paul 2007-09-28 17:52:03 UTC

How recently did you grab the code from git?  I committed a change on Thursday that might help.

Comment 4 Ademar Reis 2007-10-01 06:39:45 UTC

My git tree was from friday. Anyway, I tried with today's git, but no luck yet:

Starting program: /usr/games/neverball.bin 
[Thread debugging using libthread_db enabled]
[New Thread -1217902896 (LWP 5462)]
[New Thread -1220617328 (LWP 5465)]
Mesa: CPU vendor: GenuineIntel
Mesa: CPU name: Intel(R) Celeron(R) M CPU        410  @ 1.46GHz
Mesa: MMX cpu detected.
Mesa: SSE cpu detected.
Mesa: Not testing OS support for SSE, leaving enabled.
Mesa: Mesa 7.0.2 DEBUG build Oct  1 2007 10:16:05
Mesa warning: couldn't open libtxc_dxtn.so, software DXTn compression/decompression unavailable

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread -1217902896 (LWP 5462)]
0xb6ad1fc7 in quadfunc_unfilled_rgba (ctx=0x81d8f80, v0=0, v1=1, v2=2, v3=3)
    at swrast_setup/ss_tritmp.h:201
201           GLubyte ef1 = VB->EdgeFlag[v1];
(gdb) bt
#0  0xb6ad1fc7 in quadfunc_unfilled_rgba (ctx=0x81d8f80, v0=0, v1=1, v2=2, v3=3)
    at swrast_setup/ss_tritmp.h:201
#1  0xb6a4c37b in _tnl_render_quads_verts (ctx=0x81d8f80, start=0, count=4, flags=55)
    at tnl/t_vb_rendertmp.h:338
#2  0xb6a4dc33 in run_render (ctx=0x81d8f80, stage=0x821accc) at tnl/t_vb_render.c:320
#3  0xb6a40918 in _tnl_run_pipeline (ctx=0x81d8f80) at tnl/t_pipeline.c:158
#4  0xb697f38e in intelRunPipeline (ctx=0x81d8f80) at intel_tris.c:764
#5  0xb6a41845 in _tnl_draw_prims (ctx=0x81d8f80, arrays=0x8208eb8, prim=0x8207a14, nr_prims=1, 
    ib=0x0, min_index=0, max_index=3) at tnl/t_draw.c:402
#6  0xb6a36975 in vbo_exec_vtx_flush (exec=0x82078f0) at vbo/vbo_exec_draw.c:215
#7  0xb6a35427 in vbo_exec_FlushVertices (ctx=0x81d8f80, flags=1) at vbo/vbo_exec_api.c:685
#8  0xb69d0464 in _mesa_PopMatrix () at main/matrix.c:274
#9  0x080507dc in sol_back (fp=0x80772c0, n=256, f=512, t=122.220001) at share/solid.c:214
#10 0x0805a1d4 in game_draw_back (pose=<value optimized out>, d=<value optimized out>, 
    p=<value optimized out>) at ball/game.c:348
#11 0x0805ab46 in game_draw (pose=0, st=0) at ball/game.c:479
#12 0x0806486c in title_paint (id=19, st=0) at ball/st_title.c:108
#13 0x080580ff in st_paint () at share/state.c:69
#14 0x08065b2a in main (argc=Cannot access memory at address 0x1
) at ball/main.c:272
#15 0xb7c3ff90 in __libc_start_main () from /lib/i686/libc.so.6
#16 0x0804a641 in _start ()

Comment 5 Ademar Reis 2007-10-01 09:56:41 UTC

A simple check for the validity of VB->EdgeFlag workarounded the problem, without major consequences so far:

--- mesa-7.0-git-2007-10-01/src/mesa/swrast_setup/ss_tritmp.h.orig   2007-09-29 15:01:47.000000000 -0300
+++ mesa-7.0-git-2007-10-01/src/mesa/swrast_setup/ss_tritmp.h   2007-10-01 13:40:17.000000000 -0300
@@ -198,6 +198,8 @@ static void TAG(quadfunc)( GLcontext *ct
 {
    if (IND & SS_UNFILLED_BIT) {
       struct vertex_buffer *VB = &TNL_CONTEXT(ctx)->vb;
+         if (!VB->EdgeFlag)
+                 return;
       GLubyte ef1 = VB->EdgeFlag[v1];
       GLubyte ef3 = VB->EdgeFlag[v3];
       VB->EdgeFlag[v1] = 0;

Comment 6 Claudio Matsuoka 2007-10-02 12:40:26 UTC

The following patch is needed to fix a problem in generic_read_RGBA_span_RGB565_MMX (x86/read_rgba_span_x86.S) that causes a crash with Neverball and Metisse in the ATI 9250 card after the previous workaround is applied:

diff -rud a/Mesa-7.0.1-mdv/src/mesa/x86/read_rgba_span_x86.S Mesa-7.0.1-mdv/src/mesa/x86/read_rgba_span_x86.S
--- a/Mesa-7.0.1-mdv/src/mesa/x86/read_rgba_span_x86.S  2007-06-21 19:10:55.000000000 -0300
+++ Mesa-7.0.1-mdv/src/mesa/x86/read_rgba_span_x86.S    2007-10-02 15:36:32.000000000 -0300
@@ -587,17 +587,17 @@
        movq    prescale, %mm6
        movq    scale, %mm7
  */
-       pushl   MASK_565_H
-       pushl   MASK_565_L
+       pushl   $MASK_565_H
+       pushl   $MASK_565_L
        movq    (%esp), %mm5
-       pushl   PRESCALE_H
-       pushl   PRESCALE_L
+       pushl   $PRESCALE_H
+       pushl   $PRESCALE_L
        movq    (%esp), %mm6
-       pushl   SCALE_H
-       pushl   SCALE_L
+       pushl   $SCALE_H
+       pushl   $SCALE_L
        movq    (%esp), %mm7
-       pushl   ALPHA_H
-       pushl   ALPHA_L
+       pushl   $ALPHA_H
+       pushl   $ALPHA_L
        movq    (%esp), %mm3
        addl    $32,%esp

Comment 7 Claudio Matsuoka 2007-10-03 03:52:56 UTC

Some extra information available at http://helllabs.org/blog/20071002/mesa-quiz-spot-the-bug-with-patch

Comment 8 Brian Paul 2007-11-05 13:45:07 UTC

I've committed both patches.

Comment 9 Adam Jackson 2009-08-24 12:28:08 UTC

Mass version move, cvs -> git

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.