Bug 95085

Test with Mesa-git 11.3.0-devel (from Padoka PPA) on Ubuntu 16.04, results identical, no changes.

Had to mention that this shader is used to batch multiple ninepatch/image/text draws using minimal count of draw calls.

Observation: works with llvmpipe, but not with softpipe. My personal bet, not rooted in reality, is that something goes wrong with the pow(foo, 1/x) bit. Something becomes NaN or inf or whatever...

The actual shader, btw, appears to be:

#version 330 core

smooth in vec2 vTex2C;
smooth in vec4 vFiltColor;
flat in uint nModeOut;

out vec4 fragColor;

uniform sampler2D tNinepatch;
uniform sampler2D tFonttex;

void main() {
        if (nModeOut == 0U) {
                fragColor = textureLod(tNinepatch, vTex2C.xy, 0) * vFiltColor;
        }
        else {
                float fIntensity = max(vFiltColor.r, max(vFiltColor.g, vFiltColor.b)) * 0.6 + 1.4;
                float fAlpha = pow(textureLod(tFonttex, vTex2C.xy, 0).r, 1.0 / fIntensity);
                fragColor = vec4(1.0, 1.0, 1.0, fAlpha) * vFiltColor;
        }
}

(In reply to Ilia Mirkin from comment #2)
> My personal bet,
> not rooted in reality, is that something goes wrong with the pow(foo, 1/x)
> bit. Something becomes NaN or inf or whatever...

I tried both variants with pow() and without it - results are identical - no text. I'll make apitrace without pow() and post it here.

Created attachment 123194 [details]
apitrace_no_pow

Added apitrace without pow().

(Sorry that I forgot about alpha correction using pow().)

The geometry shader appear to be broken:

----------
#version 330 core

layout (points) in;
layout (triangle_strip, max_vertices = 4) out;

uniform mat4 mOrtho;

in vec4 vTexCoord[];
in vec4 vFilterColor[];
in uint nMode[];

smooth out vec2 vTex2C;
smooth out vec4 vFiltColor;
flat out uint nModeOut;

void main () {
	vec4 vPos = gl_in[0].gl_Position;
	vFiltColor = vFilterColor[0];
	nModeOut = nMode[0];

    gl_Position = mOrtho * vec4(vPos.x, vPos.y, 0, 1);
	vTex2C = vec2(vTexCoord[0].x, vTexCoord[0].y);    
    EmitVertex();

	gl_Position = mOrtho * vec4(vPos.x, vPos.w, 0, 1);
    vTex2C = vec2(vTexCoord[0].x, vTexCoord[0].w);    
    EmitVertex();

	gl_Position = mOrtho * vec4(vPos.z, vPos.y, 0, 1);
    vTex2C = vec2(vTexCoord[0].z, vTexCoord[0].y);    
    EmitVertex();

	gl_Position = mOrtho * vec4(vPos.z, vPos.w, 0, 1);
    vTex2C = vec2(vTexCoord[0].z, vTexCoord[0].w);    
    EmitVertex();
}
---------------------

Note that it only sets vFiltColor and nModeOut once, at the top, thus are only defined for the first EmitVertex(). However vFiltColor has to be set for every vertex, and nModeOut has to be set for the provoking vertex, which is GL_LAST_VERTEX_CONVENTION at the time of the draw @256 in the trace. 

From the spec, the definition of EmitVertex is:

"""
Emit the current values of output variables to the current output primitive.
On return from this call, the values of output variables are undefined.
"""

I believe that it happens to work out on i965 because it just happens to keep the old value.

Thank for your help Ilia, tomorrow I'll make changes to geometry shader and post here the results.

Created attachment 123235 [details]
apitrace_geom.xz

I modified our geometry shader, and it doesn't work anyway.

I set vFiltColor for each output vertex, and for nModeOut I tried three variants:
1) setting it for provoking vertexes 3rd and 4th (counting from 1)
2) setting it for each output vertex
3) breaking triangle strip into two triangles using EndPrimitive() command and setting nModeOut for 3rd and 6th vertexes.

Result is same as before modifications - black boxes.
Attaching apitrace with modified geometry shader.

And again, thanks for your help Ilia.

(In reply to Leonid Maksymchuk from comment #9)
> I modified our geometry shader, and it doesn't work anyway.
> 
> I set vFiltColor for each output vertex, and for nModeOut I tried three
> variants:
> 1) setting it for provoking vertexes 3rd and 4th (counting from 1)
> 2) setting it for each output vertex
> 3) breaking triangle strip into two triangles using EndPrimitive() command
> and setting nModeOut for 3rd and 6th vertexes.
> 
> Result is same as before modifications - black boxes.
> Attaching apitrace with modified geometry shader.
> 
> And again, thanks for your help Ilia.

The most common way of "fixing" this is to just set it for each vertex. I believe each of your variants are correct though.

Replaying your (updated) trace on nv50, nvc0, llvmpipe, softpipe -- renders correctly on all them. What hardware are you testing on?

(In reply to Ilia Mirkin from comment #10)
> Replaying your (updated) trace on nv50, nvc0, llvmpipe, softpipe -- renders
> correctly on all them. What hardware are you testing on?

Currently I have Radeon HD 7770 (radeonsi) and Radeon HD 4890 (r600g) in hands, both of them renders black boxes instead of text.
Later I'll test on NV hardware.

It seems like radeonsi and r600g still have a bug in compiler.

Frankly, I don't see how your sample is supposed to render text. The fragment shader only accesses the font texture when nModeOut != 0u, and the value is just passed through from the vertex buffer. According to qapitrace, nModeIn is equal to 0 for all vertices in the final glDrawArrays call... at least based on the apitrace, I'm pretty sure that this is not a driver bug.

I spoke too soon. There seem to be several bugs here (one pointed out by Bas Nieuwenhuizen), I'm going to investigate this again in more detail.

I think I understand now where the bug is coming from, and I'm not sure if it's a driver or an application bug :)

The trace uses glVertexAttribPointer instead of glVertexAttribIPointer to set up a vertex element that will consumed by an uint GLSL variable. I kind of suspect that that should be undefined behaviour, but I haven't found a corresponding spec reference.

In any case, it looks like this causes different parts of the driver and the various auxiliary tools to disagree about whether the element should be treated as an int or as a float, and some conversion happens somewhere that makes it all go wrong...

(In reply to Nicolai Hähnle from comment #14)

> The trace uses glVertexAttribPointer instead of glVertexAttribIPointer to
> set up a vertex element that will consumed by an uint GLSL variable. I kind
> of suspect that that should be undefined behaviour, but I haven't found a
> corresponding spec reference.

I can't see how that could work (I don't understand how that could work with some drivers even). glVertexAttribPointer() means the data is really floats - if you'd use a non-float type to specify it, it will implicitly get converted to floats. So ok you could cheat there which would leave the data untouched (I haven't looked at the trace), and rely on the uint input to just read that data as-is maybe. Not sure if that could work even if you could trick the implementation to do this - integers as floats have some bad habits, namely all your negative numbers tend to change their value (as they are NaNs), and your positive numbers may have some attraction to zero (as they are denorms...).
In any case, just don't do this - these different attrib pointers exist for a reason. Also see http://stackoverflow.com/questions/28014864/why-different-variations-of-glvertexattribpointer-do-exist

The shader only tests for non-zeroness, so it's possible that the difference between radeonsi and the other drivers is that a denorm flush happens somewhere with radeonsi and not the other drivers.

Anyway, I'm feeling lucky and closing this as NOTOURBUG for the second time today :)

FYI, the DX10_CLAMP shader bit flushes NaNs to 0. All other floating-point shader opcodes flush denorms to 0. 32-bit denorms can be enabled with a significant performance cost by setting conf->float_mode (see how it's done for FP64).

Sorry for bothering You with our bugs.
Next time I'll review the code more precisely, before reporting anything.

Thanks everyone for your help.