Bug 107772

Summary: Mesa preprocessor matches if(def)s & endifs incorrectly
Product: Mesa Reporter: Eero Tamminen <eero.t.tamminen>
Component: glsl-compilerAssignee: Timothy Arceri <t_arceri>
Status: VERIFIED FIXED QA Contact: Intel 3D Bugs Mailing List <intel-3d-bugs>
Severity: normal    
Priority: medium CC: idr, t_arceri
Version: gitKeywords: bisected, regression
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Eero Tamminen 2018-08-31 14:26:10 UTC
Mesa started to give bogus error:
  0:10(1): preprocessor error: Unterminated #if

for GfxBench ALU2 test shader between these commits:
2018-08-29 17:51:11 8fb966688b: st/mesa: Disable blending for integer formats
2018-08-30 16:41:50 d9cf4308ce: i965/screen: Allow modifiers on sRGB formats

Test-case:
* bin/testfw_app --gfx glfw --gl_api desktop_core --width 1920 --height 1080 --test_id gl_alu2

GfxBench is proprietary, so I can't provide whole shader, but these are the pre-processor directive lines from the failing shader:
-----------------------------------
[ERROR]: 1: #version 400 core
[ERROR]: 2: #define SV_30    1
[ERROR]: 3: #define highp
[ERROR]: 4: #define mediump
[ERROR]: 5: #define lowp
[ERROR]: 6: #define TYPE_vertex
[ERROR]: 7: #define DO_EMISSIVE_AND_AMBIENT 0
[ERROR]: 8: #define NUM_COLORS 64
[ERROR]: 9: #define NUM_LIGHTS 16
[ERROR]: 10: #ifdef TYPE_fragment
[ERROR]: 20: #if !DO_EMISSIVE_AND_AMBIENT
[ERROR]: 25: #define DO_LOOP_ITERATION(i) \
[ERROR]: 46: #define DO_2_LOOP_ITERATIONS(i) \
[ERROR]: 50: #define DO_4_LOOP_ITERATIONS(i) \
[ERROR]: 54: #define DO_6_LOOP_ITERATIONS(i) \
[ERROR]: 58: #define DO_8_LOOP_ITERATIONS(i) \
[ERROR]: 62: #define DO_16_LOOP_ITERATIONS(i) \
[ERROR]: 66: #endif
[ERROR]: 69: #if !DO_EMISSIVE_AND_AMBIENT
[ERROR]: 71: #endif
[ERROR]: 76: #if DO_EMISSIVE_AND_AMBIENT
[ERROR]: 85: #else
[ERROR]: 104: #endif
[ERROR]: 106: #endif
[ERROR]: 108: #ifdef TYPE_vertex
[ERROR]: 112: #if !DO_EMISSIVE_AND_AMBIENT
[ERROR]: 115: #endif
[ERROR]: 120: #if !DO_EMISSIVE_AND_AMBIENT
[ERROR]: 122: #endif
[ERROR]: 125: #endif
-----------------------------------

One should also be able to reproduce this with the GUI-only free version of GfxBench v4 from gfxbench.com, but I haven't tested that.

(No other tests we run were affected.)

I haven't bisected this, but the most likely candidate looks this:
  28a3731e3f: glsl: skip stringification in preprocessor if in unreachable branch
Comment 1 Ian Romanick 2018-08-31 15:51:40 UTC
I'll add a piglit test for this.  I cut-and-paste that code into a shader (and did 's/[[]ERROR]: [0-9]*: //'), and I get:

Failed to compile vertex shader bad.vert: 0:30(1): error: syntax error, unexpected $end
Comment 2 Ian Romanick 2018-08-31 15:52:41 UTC
(In reply to Ian Romanick from comment #1)
> I'll add a piglit test for this.  I cut-and-paste that code into a shader
> (and did 's/[[]ERROR]: [0-9]*: //'), and I get:

And 's/ \\$//'

> Failed to compile vertex shader bad.vert: 0:30(1): error: syntax error,
> unexpected $end
Comment 3 Ian Romanick 2018-08-31 16:00:25 UTC
(In reply to Ian Romanick from comment #1)
> I'll add a piglit test for this.  I cut-and-paste that code into a shader
> (and did 's/[[]ERROR]: [0-9]*: //'), and I get:
> 
> Failed to compile vertex shader bad.vert: 0:30(1): error: syntax error,
> unexpected $end

Which I now realized is due to the shader being empty after preprocessing.  With the test case below, I cannot reproduce this issue.  I tried a couple similar things.  Eero: Can you provide a small case that reproduces this?

#version 400 core
#define SV_30    1
#define highp
#define mediump
#define lowp
#define TYPE_vertex
#define DO_EMISSIVE_AND_AMBIENT 0
#define NUM_COLORS 64
#define NUM_LIGHTS 16
#ifdef TYPE_fragment
#if !DO_EMISSIVE_AND_AMBIENT
#define DO_LOOP_ITERATION(i)
#define DO_2_LOOP_ITERATIONS(i)
#define DO_4_LOOP_ITERATIONS(i)
#define DO_6_LOOP_ITERATIONS(i)
#define DO_8_LOOP_ITERATIONS(i)
#define DO_16_LOOP_ITERATIONS(i)
#endif
#if !DO_EMISSIVE_AND_AMBIENT
#endif
#if DO_EMISSIVE_AND_AMBIENT
#else
#endif
#endif
#ifdef TYPE_vertex
#if !DO_EMISSIVE_AND_AMBIENT
#endif
#if !DO_EMISSIVE_AND_AMBIENT
#endif
#endif

void main()
{
}
Comment 4 Eero Tamminen 2018-08-31 16:27:59 UTC
That shader is in internal shader-db as "4.shader_test".

(All GfxBench shaders are a bit weird, the same shader contains both vertex & fragment shader code and it uses define & ifdef so that pre-processor extracts correct code for give shader stage for the compiler.)
Comment 5 Ian Romanick 2018-08-31 17:43:46 UTC
Here's a minimal reproducing test case:

#version 110

#ifdef THING_1
#define THING_2(x) \
                if (x)
#endif

void main()
{
    gl_Position = vec4(0);
}
Comment 6 Timothy Arceri 2018-09-01 13:59:28 UTC
Proposed patch:

https://patchwork.freedesktop.org/patch/246834/
Comment 7 Timothy Arceri 2018-09-06 02:41:56 UTC
Fixed by:

commit b9fe8ff23dcfe4956be1eac4de0838d4a3720315
Author: Timothy Arceri <tarceri@itsqueeze.com>
Date:   Sat Sep 1 23:57:38 2018 +1000

    glsl: fixer lexer for unreachable defines
    
    If we have something like:
    
       #ifdef NOT_DEFINED
       #define A_MACRO(x) \
            if (x)
       #endif
    
    The # on the #define is not skipped but the define itself is so
    this then gets recognised as #if.
    
    Until 28a3731e3f this didn't happen because we ended up in
    <HASH>{NONSPACE} where BEGIN INITIAL was called stopping the
    problem from happening.
    
    This change makes sure we never call RETURN_TOKEN_NEVER_SKIP for
    if/else/endif when processing a define.
    
    Cc: Ian Romanick <idr@freedesktop.org>
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=107772
    Tested-By: Eero Tamminen <eero.t.tamminen@intel.com>
    Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Comment 8 Eero Tamminen 2018-09-07 15:45:19 UTC
Verified. GfxBench ALU2 test works fine now like the other tests.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.