Backtrace of X crashing:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f5a5b8e86f0 (LWP 6247)]
0x00007f5a4efdaba1 in fetch_texel (ctx=0xeb4870, texcoord=0x7fff639056f0,
    lambda=0, unit=0, color=0x7fff63905720) at s_fragprog.c:45
45      s_fragprog.c: No such file or directory.
        in s_fragprog.c
(gdb) bt
#0  0x00007f5a4efdaba1 in fetch_texel (ctx=0xeb4870, texcoord=0x7fff639056f0,
    lambda=0, unit=0, color=0x7fff63905720) at s_fragprog.c:45
#1  0x00007f5a4f02682b in _mesa_execute_program (ctx=0xeb4870,
    program=0x11899b0, machine=0xee0490) at prog_execute.c:326
#2  0x00007f5a4efdada3 in _swrast_exec_fragment_program (ctx=0xeb4870,
    span=0x7fff63909990) at s_fragprog.c:156
#3  0x00007f5a4efa14e5 in _swrast_write_rgba_span (ctx=0xeb4870,
    span=0x7fff63909990) at s_span.c:1552
#4  0x00007f5a4efcc11e in general_triangle (ctx=0xeb4870,
    v0=<value optimized out>, v1=<value optimized out>, v2=0x7f5a4e09c440)
    at s_tritemp.h:1142
#5  0x00007f5a4efec5ac in quadfunc_rgba (ctx=0xeb4870, v0=0, v1=0, v2=2,
    v3=1670403840) at ss_tritmp.h:210
#6  0x00007f5a4f01002d in _tnl_render_quads_verts (ctx=0xeb4870,
    start=<value optimized out>, count=4, flags=<value optimized out>)
    at t_vb_rendertmp.h:338
#7  0x00007f5a4f0110e7 in run_render (ctx=0xeb4870,
    stage=<value optimized out>) at t_vb_render.c:320
#8  0x00007f5a4f0051ff in _tnl_run_pipeline (ctx=0xeb4870) at t_pipeline.c:158
#9  0x00007f5a4eff450e in _tnl_draw_prims (ctx=0xeb4870,
    arrays=<value optimized out>, prim=0xee1dc4, nr_prims=1, ib=0x0,
    min_index=<value optimized out>, max_index=3) at t_draw.c:402
#10 0x00007f5a4f0509b2 in vbo_exec_vtx_flush (exec=0xee1b80)
    at vbo_exec_draw.c:215
#11 0x00007f5a4f04ce38 in vbo_exec_FlushVertices (ctx=<value optimized out>,
    flags=0) at vbo_exec_api.c:700
#12 0x00007f5a4ef27c1e in _mesa_Finish () at context.c:1679
#13 0x00007f5a586437cd in __glXDisp_SwapBuffers (cl=0xc99010,
    pc=<value optimized out>) at glxcmds.c:1495
#14 0x00007f5a58646945 in __glXDispatch (client=0x8828e0) at glxext.c:561
#15 0x000000000044f942 in Dispatch () at dispatch.c:502
#16 0x0000000000436585 in main (argc=3, argv=0x7fff6390ac48,
    envp=<value optimized out>) at main.c:452

xine --verbose 5:

...

main: probing <xv> video output plugin
video_out_xv: Xv extension is present but I couldn't find a usable yuv12 port.
              Looks like your graphics hardware driver doesn't support Xv?!
main: probing <opengl> video output plugin
video_out_opengl: setup of '2D_Tex_Fragprog'
video_out_opengl: extension GL_EXT_bgra: OK
video_out_opengl: extension GL_EXT_texture_object: OK
video_out_opengl: extension GL_ARB_fragment_program: OK
video_out_opengl: extension GL_ARB_pixel_buffer_object: missing
main: probing <alsa> audio output plugin

...

gui_xine_open_and_play():
        mrl: 'file:/usr/share/xine/skins/xine-ui_logo.png',
        sub 'NONE',
        start_pos 0, start_time 0, av_offset 0, spu_offset 0.
xine: found input plugin  : file input plugin
xine: found demuxer plugin: image demux plugin
av_offset=0 pts
spu_offset=0 pts
video_out_opengl: setup of '2D_Tex_Fragprog'
Fragment program only supported for YV12 data
video_out_opengl: rendering '2D_Tex_Fragprog' failed, switching to fallback
video_out_opengl: setup of '2D_Tex'
XIO:  fatal IO error 11 (Resource temporarily unavailable) on X server ":0.0"
      after 2901 requests (2884 known processed) with 55 events remaining.

Is DRI loaded or not? I know you're hitting the software path, but having DRI loaded or not can make subtle differences...

I used the radeonhd driver when the crash happened so I think there was no dri loaded.

Matthias, loading the "dri" module is the default now.

Yes, you're right.


tank:~ # cat /var/log/Xorg.0.log
...
(II) "dri" will be loaded by default.
...
(II) LoadModule: "dri"
(II) Loading /usr/lib64/xorg/modules//extensions/libdri.so
(II) Module dri: vendor="X.Org Foundation"
        compiled for 1.4.0.90, module version = 1.0.0
        ABI class: X.Org Server Extension, version 0.3
(II) Loading extension XFree86-DRI
...

I've seen a similar backtrace with Mesa 6.5.2 (without dri). In that case it was caused by a fragment shader trying to fetch texels from a texture unit that didn't have any texture assigned to it.

I kluged around it with:

+   if (swrast->TextureSample[unit] == NULL)
+   {
+     _mesa_printf("%s: texture unit %d is NULL, ignoreing\n", __FUNCTION__,
+         unit);
+         color[0] = 1.0;
+         color[1] = 0.0;
+         color[2] = 0.0;
+         color[3] = 1.0;
+         return;
+   }
+

in fetch_texel in s_nvfragprog.c (the location of that function has changed in newer sources).

Would be worth trying something like this patch to see if that helps.

if you have a core dump, or can reproduce the crash, then at the gdb prompt typeing:

select 0
info frame
print unit
print swrast->TextureSample[unit]

and paste the results in here.

I don't know if xine's doing the same thing, can you get the actual fragment program out of it?

(In reply to comment #7)
> I've seen a similar backtrace with Mesa 6.5.2 (without dri). In that case it
> was caused by a fragment shader trying to fetch texels from a texture unit that
> didn't have any texture assigned to it.

Thanks for analyzing. Two points:

- If setting up a texture fails, the fragment shader path shouldn't be taken. This *could* be xine's OpenGL plugin fault, but my guess is that texture upload in Mesa runs into a subtle issue and leaves the texture undefined.

- Mesa shouldn't crash, if an undefined texture unit is accessed.

> I kluged around it with:
> 
> +   if (swrast->TextureSample[unit] == NULL)
> +   {
> +     _mesa_printf("%s: texture unit %d is NULL, ignoreing\n", __FUNCTION__,
> +         unit);
> +         color[0] = 1.0;
> +         color[1] = 0.0;
> +         color[2] = 0.0;
> +         color[3] = 1.0;
> +         return;
> +   }

Doesn't sound too bad from my view. You can't do much more.

> I don't know if xine's doing the same thing, can you get the actual fragment
> program out of it?


The fragment program is the following:

!!ARBfp1.0
ATTRIB tex = fragment.texcoord[0];
PARAM  off = program.env[0];
TEMP u, v;
TEMP res, tmp;
ADD u, tex, off.xwww;
TEX res, u, texture[0], 2D;
MUL v, tex, .5;
ADD u, v, off.xyww;
ADD v, v, off.zyww;
TEX tmp.x, u, texture[0], 2D;
MAD res, res, 1.164, -0.073; /* -1.164*16/255 */
TEX tmp.y, v, texture[0], 2D;
SUB tmp, tmp, { .5, .5 };
MAD res, { 0, -.391, 2.018 }, tmp.xxxw, res;
MAD result.color, { 1.596, -.813, 0 }, tmp.yyyw, res;
END


The images are loaded with

  glTexSubImage2D (GL_TEXTURE_2D, 0,  1, 0,  frame->width, frame->height,
                   GL_LUMINANCE, GL_UNSIGNED_BYTE, frame->vo_frame.base[0]);
  glTexSubImage2D (GL_TEXTURE_2D, 0,  1, frame->height+2,  w2, h2,
                   GL_LUMINANCE, GL_UNSIGNED_BYTE, frame->vo_frame.base[1]);
  glTexSubImage2D (GL_TEXTURE_2D, 0,  w2+2, frame->height+2,  w2, h2,
                   GL_LUMINANCE, GL_UNSIGNED_BYTE, frame->vo_frame.base[2]);

into a single preallocated texture with the following layout:

/* YUV texture layout                   .llllll.
 *                                      .llllll.
 * lum size 6x4 ->                      .llllll.
 *                                      .llllll.
 * empty w/ 0   (box filter y)          .......
 * empty w/ 0.5 (box filter u+v)        /////////
 *                                      /uuu/vvv/
 * u, v size 3x2 each ->                /uuu/vvv/
 *                                      ///////// */


Texture allocation is done by:

        this->glBindTextureEXT (GL_TEXTURE_2D, 0);
      glTexParameteri (GL_TEXTURE_2D,  GL_TEXTURE_MAG_FILTER, GL_LINEAR);
      glTexParameteri (GL_TEXTURE_2D,  GL_TEXTURE_MIN_FILTER, GL_LINEAR);
      glTexImage2D (GL_TEXTURE_2D, 0, glformat, tex_w, tex_h,
                      0, texformat, GL_UNSIGNED_BYTE, tmp);

and glGetError() is checked. tex_w and tex_h are powers of two. Of course, the texture could be too large to handle, the glGetError() should be able to find that.

Created attachment 17572 [details] [review]
patch to check for null texObj ptr

Could you try the attached patch (against either 7.0.3 or 7.1-rc1 or git)?

Sorry for the late response. I'll try your patch as soon as I find some time (probably next week).

I tested your patch against 7.0.3 and the bug is still reproducible. Feel free to ask if you need more information.

I forgot to mention that it seems to work fine with Mesa 7.1RC1 (without your patch).

Mesa 7.2 has been released some time ago, so this issue should be fixed meanwhile.