Bug 35073

mesa 7.10
xf86-video-intel 2.14.0
libdrm 2.4.23
xorg-server 1.9.4
Mesa DRI Mobile Intel® GM45 Express Chipset GEM 20100330 DEVELOPMENT

1) Install 0 A.D. r9042 or later (http://trac.wildfiregames.com/wiki/BuildInstructions)
2) Launch it, select "single player", select "Belgian Bog", "start game".
3) If shadows are disabled by default, press F10, "settings", "enable shadows".
4) Observe that trees cast shadows which match the leaf texture pattern.
5) Press F9 to open the console.
6) Enter the command:  renderer.shadowAlphaFix=false

Expected result: No change in the rendering.

Actual result: The shadows disappear entirely for the alpha-textured polygons.

If I use the Gallium software renderer ("Gallium 0.4 on llvmpipe") then the opposite problem occurs: shadows are drawn for the whole of the alpha-textured polygons, including parts with alpha=0.

If I use the classic software renderer then it works correctly as expected. NVIDIA drivers on Windows work correctly. I assume most other drivers work correctly because I don't remember seeing people complain about this bug in the game.

The game renders shadows onto an FBO with a GL_DEPTH_ATTACHMENT. If shadowAlphaFix is false (which was the behaviour before I added this workaround), the FBO has no other attachments, and it has glDrawBuffer(GL_NONE). If shadowAlphaFix is true, a GL_ALPHA8 color attachment is added and glDrawBuffer is not called.

The transparent objects are drawn with:

    glEnable(GL_ALPHA_TEST);
    glAlphaFunc(GL_GREATER, 0.4f);
    glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_COMBINE);
    glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_RGB_ARB, GL_REPLACE);
    glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_RGB_ARB, GL_PREVIOUS);
    glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_RGB_ARB, GL_SRC_COLOR);
    glTexEnvi(GL_TEXTURE_ENV, GL_COMBINE_ALPHA_ARB, GL_REPLACE);
    glTexEnvi(GL_TEXTURE_ENV, GL_SOURCE0_ALPHA_ARB, GL_TEXTURE);
    glTexEnvi(GL_TEXTURE_ENV, GL_OPERAND0_ALPHA_ARB, GL_SRC_ALPHA);

If I change the GL_GREATER to GL_LESS, then the Intel behaviour flips when shadowAlphaFix=false: shadows are drawn for the whole alpha-textured polygon (instead of for none of it). Gallium software with shadowAlphaFix false still draws the whole polygon in this case (i.e. it doesn't flip).

In the Intel case, this looks like the fragment's alpha is being interpreted as 0 when the alpha test is applied if there is no color buffer.

The OpenGL spec looks a little unclear to me - it says the alpha test "applies only in RGBA mode, and only if the color buffer has a fixed-point
or floating-point format", but I don't see what it wants if there is no color buffer. I think the most useful and compatible behaviour would be to apply the alpha test in this case and to compute the correct alpha value first.