Bug 94448 (i965-deqp)

Summary:	double Everything's Quality, Please! (Fix all the dEQP bugs!)
Product:	Mesa	Reporter:	Kenneth Graunke <kenneth>
Component:	Drivers/DRI/i965	Assignee:	Kenneth Graunke <kenneth>
Status:	RESOLVED FIXED	QA Contact:	Intel 3D Bugs Mailing List <intel-3d-bugs>
Severity:	normal
Priority:	medium	CC:	chadversary, randy.xu, sayyarhassan
Version:	unspecified
Hardware:	All
OS:	Linux (All)
Whiteboard:
i915 platform:		i915 features:
Bug Depends on:	89636, 92358, 92743, 94080, 94116, 94127, 94196, 94198, 94449, 94450, 94451, 94452, 94453, 94454, 94455, 94456, 94458, 94485, 94621, 94668, 94681, 94713, 94714, 94715, 95138, 98131, 98132, 98133, 98134, 98135, 98136, 98159, 98223, 98241, 98242, 98243, 98244, 98245, 98246, 98248, 98249, 98250, 98522, 99076, 100650, 102999
Bug Blocks:	8945

Description Kenneth Graunke 2016-03-09 00:49:22 UTC

This is a tracker bug for i965 dEQP failures.

We want to get 100% of the tests in the "must pass" lists (deqp/android/cts/master/gles*-master.txt) passing.  

https://android.googlesource.com/platform/external/deqp

For now, we are running the x11_glx target and using windowed mode (not FBO mode).

Comment 1 Kenneth Graunke 2016-03-09 02:06:22 UTC

To run the tests, I have been doing:

./deqp-gles3 --deqp-visibility=hidden --deqp-caselist-file=../../android/cts/master/gles3-master.txt

Comment 2 Randy 2016-10-13 08:41:49 UTC

I run the dEQP test on SKL/GT4e/Ubuntu 16.04 64bit system, with the latest dEQP mesa master, and get 586 failures in total.

Mesa git top commit: 389d6dedbe75defe07216ad761569a9b94f44e58
dEQP git top commit: ca988480be945772473f9256b6ae91fa6aa62bd1

I compare these 586 failures with ones tracked here and categorized in http://whitecape.org/paste/nougat-cts.txt, it seems 27 failures not tracked yet, they are:

dEQP-GLES31.functional.debug.negative_coverage.callbacks.shader.compile_compute_shader.qpa: <Result StatusCode="Fail">Compute Shader should not have compiled with #version 300 es.</Result>
dEQP-GLES31.functional.debug.negative_coverage.get_error.buffer.copy_image_sub_data.qpa: <Result StatusCode="Fail">Incorrect error was reported</Result>
dEQP-GLES31.functional.debug.negative_coverage.get_error.buffer.draw_buffers.qpa: <Result StatusCode="Fail">Incorrect error was reported</Result>
dEQP-GLES31.functional.debug.negative_coverage.get_error.shader.compile_compute_shader.qpa: <Result StatusCode="Fail">Compute Shader should not have compiled with #version 300 es.</Result>
dEQP-GLES31.functional.debug.negative_coverage.get_error.texture.texparameterIiv.qpa: <Result StatusCode="Fail">Incorrect error was reported</Result>
dEQP-GLES31.functional.debug.negative_coverage.get_error.texture.texparameterIuiv.qpa: <Result StatusCode="Fail">Incorrect error was reported</Result>
dEQP-GLES31.functional.debug.negative_coverage.log.shader.compile_compute_shader.qpa: <Result StatusCode="Fail">Compute Shader should not have compiled with #version 300 es.</Result>
dEQP-GLES31.functional.separate_shader.validation.io_blocks.match_different_member_struct_names.qpa: <Result StatusCode="Fail">draw: glGetError() returned GL_INVALID_OPERATION at glsShaderLibraryCase.cpp:1298</Result>
dEQP-GLES31.functional.separate_shader.validation.varying.match_different_struct_names.qpa: <Result StatusCode="Fail">draw: glGetError() returned GL_INVALID_OPERATION at glsShaderLibraryCase.cpp:1298</Result>
dEQP-GLES31.functional.shaders.linkage.io_block.missing_output_block.qpa: <Result StatusCode="Fail">expected linking to fail, but passed.</Result>
dEQP-GLES31.functional.shaders.linkage.uniform.block.differing_precision.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to link.</Result>
dEQP-GLES31.functional.shaders.linkage.uniform.block.layout_qualifier_mismatch_3.qpa: <Result StatusCode="Fail">expected linking to fail, but passed.</Result>
dEQP-GLES31.functional.tessellation.user_defined_io.negative.per_patch_array_of_structs.qpa: <Result StatusCode="Fail">expected compile or link to fail, but passed.</Result>
dEQP-GLES31.functional.tessellation.user_defined_io.negative.per_patch_structs_containing_arrays.qpa: <Result StatusCode="Fail">expected compile or link to fail, but passed.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_and_file_expression_fragment.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_and_file_expression_vertex.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_expression_fragment.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_expression_vertex.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.conditional_inclusion.defined_macro_defined_test_fragment.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.conditional_inclusion.defined_macro_defined_test_vertex.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.texture_functions.invalid.textureoffset_sampler2darrayshadow_vec4_ivec2_fragment.qpa: <Result StatusCode="Fail">expected compilation to fail, but shaders compiled correctly.</Result>
dEQP-GLES3.functional.shaders.texture_functions.invalid.textureoffset_sampler2darrayshadow_vec4_ivec2_vertex.qpa: <Result StatusCode="Fail">expected compilation to fail, but shaders compiled correctly.</Result>
dEQP-GLES3.functional.state_query.integers.viewport_getfloat.qpa: <Result StatusCode="Fail">Got invalid float value</Result>
dEQP-GLES3.functional.state_query.integers.viewport_getinteger64.qpa: <Result StatusCode="Fail">Got invalid integer value</Result>
dEQP-GLES3.functional.state_query.integers.viewport_getinteger.qpa: <Result StatusCode="Fail">Got invalid integer value</Result>
dEQP-GLES3.functional.ubo.random.all_per_block_buffers.33.qpa: <Result StatusCode="Fail">Image compare failed</Result>
dEQP-GLES3.functional.ubo.single_nested_struct_array.per_block_buffer.shared_instance_array_fragment.qpa: <Result StatusCode="Fail">Image compare failed</Result>


Please help to review and comment. I will check more and may file new bugs.

Comment 3 Randy 2016-10-13 08:44:25 UTC

All the failure cases in last comment are in Android N CTS mustpass list.

Comment 4 Mark Janes 2016-10-13 22:13:47 UTC

(In reply to Randy from comment #2)
> I run the dEQP test on SKL/GT4e/Ubuntu 16.04 64bit system, with the latest
> dEQP mesa master, and get 586 failures in total.
> 
> Mesa git top commit: 389d6dedbe75defe07216ad761569a9b94f44e58
> dEQP git top commit: ca988480be945772473f9256b6ae91fa6aa62bd1
> 
> I compare these 586 failures with ones tracked here and categorized in
> http://whitecape.org/paste/nougat-cts.txt, it seems 27 failures not tracked
> yet, they are:
> 
> dEQP-GLES31.functional.debug.negative_coverage.callbacks.shader.
> compile_compute_shader.qpa: <Result StatusCode="Fail">Compute Shader should
> not have compiled with #version 300 es.</Result>
> dEQP-GLES31.functional.debug.negative_coverage.get_error.buffer.
> copy_image_sub_data.qpa: <Result StatusCode="Fail">Incorrect error was
> reported</Result>
> dEQP-GLES31.functional.debug.negative_coverage.get_error.buffer.draw_buffers.
> qpa: <Result StatusCode="Fail">Incorrect error was reported</Result>
> dEQP-GLES31.functional.debug.negative_coverage.get_error.shader.
> compile_compute_shader.qpa: <Result StatusCode="Fail">Compute Shader should
> not have compiled with #version 300 es.</Result>
> dEQP-GLES31.functional.debug.negative_coverage.get_error.texture.
> texparameterIiv.qpa: <Result StatusCode="Fail">Incorrect error was
> reported</Result>
> dEQP-GLES31.functional.debug.negative_coverage.get_error.texture.
> texparameterIuiv.qpa: <Result StatusCode="Fail">Incorrect error was
> reported</Result>
> dEQP-GLES31.functional.debug.negative_coverage.log.shader.
> compile_compute_shader.qpa: <Result StatusCode="Fail">Compute Shader should
> not have compiled with #version 300 es.</Result>
> dEQP-GLES31.functional.separate_shader.validation.io_blocks.
> match_different_member_struct_names.qpa: <Result StatusCode="Fail">draw:
> glGetError() returned GL_INVALID_OPERATION at
> glsShaderLibraryCase.cpp:1298</Result>
> dEQP-GLES31.functional.separate_shader.validation.varying.
> match_different_struct_names.qpa: <Result StatusCode="Fail">draw:
> glGetError() returned GL_INVALID_OPERATION at
> glsShaderLibraryCase.cpp:1298</Result>
> dEQP-GLES31.functional.shaders.linkage.io_block.missing_output_block.qpa:
> <Result StatusCode="Fail">expected linking to fail, but passed.</Result>
> dEQP-GLES31.functional.shaders.linkage.uniform.block.differing_precision.qpa:
> <Result StatusCode="Fail">expected shaders to compile and link properly, but
> failed to link.</Result>
> dEQP-GLES31.functional.shaders.linkage.uniform.block.
> layout_qualifier_mismatch_3.qpa: <Result StatusCode="Fail">expected linking
> to fail, but passed.</Result>
> dEQP-GLES31.functional.tessellation.user_defined_io.negative.
> per_patch_array_of_structs.qpa: <Result StatusCode="Fail">expected compile
> or link to fail, but passed.</Result>
> dEQP-GLES31.functional.tessellation.user_defined_io.negative.
> per_patch_structs_containing_arrays.qpa: <Result StatusCode="Fail">expected
> compile or link to fail, but passed.</Result>
> dEQP-GLES3.functional.shaders.preprocessor.builtin.
> line_and_file_expression_fragment.qpa: <Result StatusCode="Fail">expected
> shaders to compile and link properly, but failed to compile.</Result>
> dEQP-GLES3.functional.shaders.preprocessor.builtin.
> line_and_file_expression_vertex.qpa: <Result StatusCode="Fail">expected
> shaders to compile and link properly, but failed to compile.</Result>
> dEQP-GLES3.functional.shaders.preprocessor.builtin.line_expression_fragment.
> qpa: <Result StatusCode="Fail">expected shaders to compile and link
> properly, but failed to compile.</Result>
> dEQP-GLES3.functional.shaders.preprocessor.builtin.line_expression_vertex.
> qpa: <Result StatusCode="Fail">expected shaders to compile and link
> properly, but failed to compile.</Result>
> dEQP-GLES3.functional.shaders.preprocessor.conditional_inclusion.
> defined_macro_defined_test_fragment.qpa: <Result StatusCode="Fail">expected
> shaders to compile and link properly, but failed to compile.</Result>
> dEQP-GLES3.functional.shaders.preprocessor.conditional_inclusion.
> defined_macro_defined_test_vertex.qpa: <Result StatusCode="Fail">expected
> shaders to compile and link properly, but failed to compile.</Result>
> dEQP-GLES3.functional.shaders.texture_functions.invalid.
> textureoffset_sampler2darrayshadow_vec4_ivec2_fragment.qpa: <Result
> StatusCode="Fail">expected compilation to fail, but shaders compiled
> correctly.</Result>
> dEQP-GLES3.functional.shaders.texture_functions.invalid.
> textureoffset_sampler2darrayshadow_vec4_ivec2_vertex.qpa: <Result
> StatusCode="Fail">expected compilation to fail, but shaders compiled
> correctly.</Result>
> dEQP-GLES3.functional.state_query.integers.viewport_getfloat.qpa: <Result
> StatusCode="Fail">Got invalid float value</Result>
> dEQP-GLES3.functional.state_query.integers.viewport_getinteger64.qpa:
> <Result StatusCode="Fail">Got invalid integer value</Result>
> dEQP-GLES3.functional.state_query.integers.viewport_getinteger.qpa: <Result
> StatusCode="Fail">Got invalid integer value</Result>
> dEQP-GLES3.functional.ubo.random.all_per_block_buffers.33.qpa: <Result
> StatusCode="Fail">Image compare failed</Result>
> dEQP-GLES3.functional.ubo.single_nested_struct_array.per_block_buffer.
> shared_instance_array_fragment.qpa: <Result StatusCode="Fail">Image compare
> failed</Result>
> 
> 
> Please help to review and comment. I will check more and may file new bugs.

Please do not use SKLGT4e as a test platform until the kernel is known to work properly.  Do not report bugs against this platform until we have it stable in the Mesa CI.

Comment 5 Matt Turner 2016-10-13 22:21:09 UTC

(In reply to Randy from comment #2)

This is a tracker. Do not comment here.

Comment 6 Randy 2016-10-14 01:06:20 UTC

(In reply to Matt Turner from comment #5)
> (In reply to Randy from comment #2)
> 
> This is a tracker. Do not comment here.

OK, please give me a chance to answer Mark's question in comment 4 :)

I verified all 27 non-tracked failure on HSW/GT2, 11 can be reproduced, and 2 are "NotSupported"

dEQP-GLES31.functional.shaders.linkage.io_block.missing_output_block.qpa: <Result StatusCode="Fail">expected linking to fail, but passed.</Result>
dEQP-GLES31.functional.shaders.linkage.uniform.block.differing_precision.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to link.</Result>
dEQP-GLES31.functional.shaders.linkage.uniform.block.layout_qualifier_mismatch_3.qpa: <Result StatusCode="Fail">expected linking to fail, but passed.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_and_file_expression_fragment.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_and_file_expression_vertex.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_expression_fragment.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.builtin.line_expression_vertex.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.conditional_inclusion.defined_macro_defined_test_fragment.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.preprocessor.conditional_inclusion.defined_macro_defined_test_vertex.qpa: <Result StatusCode="Fail">expected shaders to compile and link properly, but failed to compile.</Result>
dEQP-GLES3.functional.shaders.texture_functions.invalid.textureoffset_sampler2darrayshadow_vec4_ivec2_fragment.qpa: <Result StatusCode="Fail">expected compilation to fail, but shaders compiled correctly.</Result>
dEQP-GLES3.functional.shaders.texture_functions.invalid.textureoffset_sampler2darrayshadow_vec4_ivec2_vertex.qpa: <Result StatusCode="Fail">expected compilation to fail, but shaders compiled correctly.</Result>

dEQP-GLES31.functional.debug.negative_coverage.get_error.texture.texparameterIiv.qpa: <Result StatusCode="NotSupported">glTexParameterIiv is not supported. at es31fNegativeTextureApiTests.cpp:2231</Result>
dEQP-GLES31.functional.debug.negative_coverage.get_error.texture.texparameterIuiv.qpa: <Result StatusCode="NotSupported">glTexParameterIuiv is not supported. at es31fNegativeTextureApiTests.cpp:2341</Result>

Comment 7 Randy 2016-10-20 07:09:22 UTC

Sorry, I just know there is another mega bug 98315 to track the dEQP EGL failure, so I update the list.

Comment 8 Tapani Pälli 2018-10-24 05:31:22 UTC

As requested, everything's quality was doubled! Resolving as fixed, we can treat upcoming failures as individual bugs.

Comment 9 sayyarhassan@gmail.com (Spammer; Account disabled) 2019-03-14 08:01:00 UTC

reyrtuytyukh

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.