Created attachment 35359 [details]
Xorg log

I've tried
Section "ServerFlags"
...
Option "GlxVisuals" "all"
...
EndSection

but it's still the same.

Maxim provided a stack trace:

Breakpoint 1, st_texture_create (st=0x991ec0, target=PIPE_TEXTURE_2D, format=PIPE_FORMAT_I8_UNORM, last_level=0, width0=512, height0=32, depth0=1, bind=4) at state_tracker/st_texture.c:64
64	{
#0  st_texture_create (st=0x991ec0, target=PIPE_TEXTURE_2D, format=PIPE_FORMAT_I8_UNORM, last_level=0, width0=512, height0=32, depth0=1, bind=4) at state_tracker/st_texture.c:64
#1  0x00007ffff3409224 in reset_cache (st=0x991ec0) at state_tracker/st_cb_bitmap.c:542
#2  0x00007ffff33d519e in st_create_context_priv (pipe=0x900370, visual=<value optimized out>, share=<value optimized out>) at state_tracker/st_context.c:149
#3  st_create_context (pipe=0x900370, visual=<value optimized out>, share=<value optimized out>) at state_tracker/st_context.c:219
#4  0x00007ffff3398bd7 in st_api_create_context (stapi=<value optimized out>, smapi=0x9123e0, visual=0x7fffffffd2a0, shared_stctxi=0x0) at state_tracker/st_manager.c:589
#5  0x00007ffff32a0d5e in dri_create_context (api=<value optimized out>, visual=0x931740, cPriv=<value optimized out>, sharedContextPrivate=<value optimized out>) at dri_context.c:79
#6  0x00007ffff329aeda in dri2CreateNewContextForAPI (screen=0x912170, api=0, config=0x931740, shared=<value optimized out>, data=0x200) at ../../../../src/mesa/drivers/dri/common/dri_util.c:662
#7  0x00007ffff63be1c9 in dri2CreateContext (psc=0x900090, mode=0x904700, gc=0x900160, shareList=<value optimized out>, renderType=<value optimized out>) at dri2_glx.c:150
#8  0x00007ffff6395905 in CreateContext (dpy=0x64ed00, generic_id=33, fbconfig=<value optimized out>, shareList=0x0, allowDirect=<value optimized out>, code=3, renderType=32788, screen=0) at glxcmds.c:403
#9  0x00007ffff6396e38 in glXCreateContext (dpy=0x64ed00, vis=0x8f3860, shareList=0x0, allowDirect=1) at glxcmds.c:527
#10 0x00000000004196ff in addScreen ()
#11 0x0000000000412334 in addDisplay ()
#12 0x000000000040d8dc in main ()

Breakpoint 1, st_texture_create (st=0x991ec0, target=PIPE_TEXTURE_2D, format=PIPE_FORMAT_B8G8R8X8_UNORM, last_level=0, width0=0, height0=0, depth0=0, bind=6) at state_tracker/st_texture.c:64
64	{
#0  st_texture_create (st=0x991ec0, target=PIPE_TEXTURE_2D, format=PIPE_FORMAT_B8G8R8X8_UNORM, last_level=0, width0=0, height0=0, depth0=0, bind=6) at state_tracker/st_texture.c:64
#1  0x00007ffff34174e0 in st_finalize_texture (ctx=0x9495d0, pipe=<value optimized out>, tObj=0xcbbcb0, needFlush=<value optimized out>) at state_tracker/st_cb_texture.c:1910
#2  0x00007ffff340830a in finalize_textures (st=0x991ec0) at state_tracker/st_atom_texture.c:145
#3  0x00007ffff34045d9 in st_validate_state (st=0x991ec0) at state_tracker/st_atom.c:167
#4  0x00007ffff33d5b2a in st_draw_vbo (ctx=<value optimized out>, arrays=0x995608, prims=<value optimized out>, nr_prims=<value optimized out>, ib=0x0, index_bounds_valid=0 '\000', min_index=0, max_index=3) at state_tracker/st_draw.c:577
#5  0x00007ffff33fcf84 in vbo_exec_DrawArrays (mode=7, start=0, count=<value optimized out>) at vbo/vbo_exec_array.c:525
#6  0x0000000000427c9e in ?? ()
#7  0x0000000000427afb in drawWindowTexture ()
#8  0x0000000000427013 in drawWindow ()
#9  0x00007fffef6bbd4f in ?? () from /usr/lib/compiz/libdecoration.so
#10 0x0000000000426ef9 in paintWindow ()
#11 0x00007fffefcd0988 in ?? () from /usr/lib/compiz/libmove.so
#12 0x00007fffefacaa80 in ?? () from /usr/lib/compiz/libresize.so
#13 0x0000000000428c3d in ?? ()
#14 0x00000000004299b5 in paintOutput ()
#15 0x00007fffefacaee0 in ?? () from /usr/lib/compiz/libresize.so
#16 0x0000000000410b67 in paintScreen ()
#17 0x0000000000412a35 in eventLoop ()
#18 0x000000000040d8e9 in main ()
Mesa: User error: GL_OUT_OF_MEMORY in glTexImage

Created attachment 35409 [details] [review]
a patch to try.

Can you try the attached patch?

patch doesn't help.

Same results with latest patch.

I've committed another change (2b53f4a9c674e9b02df8a06759e7a2340f257081) that should silence the spurious GL_OUT_OF_MEMORY error.  Not sure if that'll help with the overall problem though.

(In reply to comment #7)
> I've committed another change (2b53f4a9c674e9b02df8a06759e7a2340f257081) that
> should silence the spurious GL_OUT_OF_MEMORY error.  Not sure if that'll help
> with the overall problem though.

Nope, it doesn't help.

FWIW I get textures of size 0x0 in resource_create when texture_from_pixmap is used. I guess it might be the reason the buffer allocation is failing. It's reproducible via piglit/tfp.

(In reply to comment #9)
> FWIW I get textures of size 0x0 in resource_create when texture_from_pixmap is
> used. I guess it might be the reason the buffer allocation is failing. It's
> reproducible via piglit/tfp.

I can indeed reproduce the problem with nv50 on both compiz and tfp.

I've noticed that the commit changed firstImage->base.Width2(,height,depth) to stObj->width0(,height,depth)

I've enabled DBG and added one printf below :
   /* If we already have a gallium texture, check that it matches the texture
    * object's format, target, size, num_levels, etc.
    */
   if (stObj->pt) {
           printf("stobj->pt %dx%dx%d stobj %dx%dx%d firstimage %dx%dx%d\n", stObj->pt->width0, stObj->pt->height0, stObj->pt->depth0,
                           stObj->width0, stObj->height0, stObj->depth0,
                           firstImage->base.Width2, firstImage->base.Height2, firstImage->base.Depth2);

And this is the ouput with tfp :
st_NewTextureObject
st_NewTextureImage
stobj->pt 2x2x1 stobj 0x0x0 firstimage 2x2x1
Mesa: User error: GL_OUT_OF_MEMORY in glTexImage
stobj->pt 2x2x1 stobj 0x0x0 firstimage 2x2x1
st_NewTextureObject
st_NewTextureImage
st_FreeTextureImageData
stobj->pt 2x2x1 stobj 0x0x0 firstimage 2x2x1
stobj->pt 2x2x1 stobj 0x0x0 firstimage 2x2x1

I doubt this helps, but just in case...
I don't understand what stobj, stobj->pt and firstimage are respectively.

I don't understand that comment below either, but from the output above, the image dimensions don't seem to match.
   /* If both firstImage and stObj point to a texture which can contain
    * all active images, favour firstImage.  Note that because of the
    * completeness requirement, we know that the image dimensions
    * will match.
    */

Okay, seems to be finally fixed in current GIT. Thanks.

Looks like Chia-I Wu found/fixed this with commit 719f7049bb2c7f5ca886055c9cd15b2805bd8e97.  Thanks!

Confirmed, thanks Chia-I Wu !