Bug 11404

Summary:	Nexuiz GLSL shader compilation fails
Product:	Mesa	Reporter:	Rudolf Polzer <polzer>
Component:	Mesa core	Assignee:	mesa-dev
Status:	RESOLVED FIXED	QA Contact:
Severity:	major
Priority:	medium	CC:	brian.paul, idr, peter
Version:	git
Hardware:	x86-64 (AMD64)
OS:	Linux (All)
Whiteboard:
i915 platform:		i915 features:
Attachments:	default.glsl file of DarkPlaces and a version that doesn't cause SIGABRT ready-made fragment and vertex shader program for use in the fs example program nexuiz-24-12-08.log and default.glsl

Description Rudolf Polzer 2007-06-28 06:07:32 UTC

Out of curiosity, I was trying out Mesa 7.0's (and Mesa from GIT) software rasterizer with the DarkPlaces engine. The low frames per second aside (which makes this unusable anyway), I found some issues in the GLSL compiler.

The GLSL shader used by Nexuiz (I am using the current DarkPlaces engine from svn, and no specific GLSL shader, just the one inside the engine) fails to compile with the following message:

  Error: Function 'vec2' not found (check argument types)

The offending line is

  TexCoord = vec2(gl_TextureMatrix[0] * gl_MultiTexCoord0);

I tried rewriting this as

  TexCoord = (gl_TextureMatrix[0] * gl_MultiTexCoord0).st;

and changed one other occurrence of this the same way, and actually got further - it now crashes with signal 6 when compiling:

  Mesa 7.1 implementation error: invalid writemask in make_writemask()
  Please report at bugzilla.freedesktop.org
  Mesa 7.1 implementation error: invalid writemask in make_writemask()
  Please report at bugzilla.freedesktop.org
  Mesa 7.1 implementation error: invalid writemask in make_writemask()
  Please report at bugzilla.freedesktop.org
  nexuiz: shader/slang/slang_emit.c:960: emit_move: Assertion `n->Children[0]->Store->Index >= 0' failed.
  Received signal 6, exiting...

Ifdeffing out stuff till the error disappears led me to the conclusion that this is the (or one) offending line:

  color.rgb *= AmbientColor + DiffuseColor * myhalf(max(float(dot(surfacenormal, diffusenormal)), 0.0));

(myhalf is #defined above as float, so you can just ignore it)

Curiously, I also got a segmentation fault when enabling this line...

  color.rgb = color.rgb * SceneBrightness / (ContrastBoostCoeff * color.rgb + myhvec3(1, 1, 1));

but I could not reproduce it, so I can't provide a backtrace.

Would be nice if this could get fixed... maybe there are also some nonconforming things in the DarkPlaces GLSL shader, would be nice if anyone could look and help sorting that out... it currently works fine with nvidia and mostly fine (too many ALU instructions when enabling offsetmapping and a subsequent driver crash) with ATI.

Comment 1 Brian Paul 2007-07-03 10:44:05 UTC

I've fixed the vec2() constructor problem and another bug involving 'st' writemasks/swizzles.
Can you try Mesa git again and report back?

Comment 2 Rudolf Polzer 2007-07-06 16:40:46 UTC

Now I immediately get to the SIGABRT:

(gdb) bt
#0  0x00002b00b4108cab in raise () from /lib/libc.so.6
#1  0x00002b00b410a660 in abort () from /lib/libc.so.6
#2  0x00002b00b4102436 in __assert_fail () from /lib/libc.so.6
#3  0x00002b00b6bd525c in emit_move (emitInfo=0x7ffff7c6de20, n=0x387b1840)
    at shader/slang/slang_emit.c:960
#4  0x00002b00b6bd2fb9 in emit (emitInfo=0x7ffff7c6de20, n=0x33fb)
    at shader/slang/slang_emit.c:1676

This value of n looks odd.

#5  0x00002b00b6bd2c20 in emit (emitInfo=0x7ffff7c6de20, n=0x387b18b0)
    at shader/slang/slang_emit.c:1517
#6  0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387b59a8)
    at shader/slang/slang_emit.c:1516
#7  0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387b9aa0)
    at shader/slang/slang_emit.c:1516
#8  0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387b9ce8)
    at shader/slang/slang_emit.c:1516
#9  0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387b9fb8)
    at shader/slang/slang_emit.c:1516
#10 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba288)
    at shader/slang/slang_emit.c:1516
#11 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba368)
    at shader/slang/slang_emit.c:1516
#12 0x00002b00b6bd2c49 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba3d8)
    at shader/slang/slang_emit.c:1527
#13 0x00002b00b6bd2c20 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba448)
    at shader/slang/slang_emit.c:1517
#14 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba528)
    at shader/slang/slang_emit.c:1516
#15 0x00002b00b6bd4f0c in emit_move (emitInfo=0x7ffff7c6de20, n=0x387ba598)
    at shader/slang/slang_emit.c:930
#16 0x00002b00b6bd2fb9 in emit (emitInfo=0x7ffff7c6de20, n=0x33fb)
    at shader/slang/slang_emit.c:1676
#17 0x00002b00b6bd2c20 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba608)
    at shader/slang/slang_emit.c:1517
#18 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba6e8)
    at shader/slang/slang_emit.c:1516
#19 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ba918)
    at shader/slang/slang_emit.c:1516
#20 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387baad8)
    at shader/slang/slang_emit.c:1516
#21 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387bad90)
    at shader/slang/slang_emit.c:1516
#22 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387bae70)
    at shader/slang/slang_emit.c:1516
#23 0x00002b00b6bd2c49 in emit (emitInfo=0x7ffff7c6de20, n=0x387baee0)
    at shader/slang/slang_emit.c:1527
#24 0x00002b00b6bd2c20 in emit (emitInfo=0x7ffff7c6de20, n=0x387baf50)
    at shader/slang/slang_emit.c:1517
#25 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387bb030)
    at shader/slang/slang_emit.c:1516
#26 0x00002b00b6bd4f0c in emit_move (emitInfo=0x7ffff7c6de20, n=0x387bb0a0)
    at shader/slang/slang_emit.c:930
#27 0x00002b00b6bd2fb9 in emit (emitInfo=0x7ffff7c6de20, n=0x33fb)
    at shader/slang/slang_emit.c:1676
#28 0x00002b00b6bd2c20 in emit (emitInfo=0x7ffff7c6de20, n=0x387bb110)
    at shader/slang/slang_emit.c:1517
#29 0x00002b00b6bd2c20 in emit (emitInfo=0x7ffff7c6de20, n=0x387bb180)
    at shader/slang/slang_emit.c:1517
#30 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387c06d8)
    at shader/slang/slang_emit.c:1516
#31 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ebec8)
    at shader/slang/slang_emit.c:1516
#32 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387f5870)
    at shader/slang/slang_emit.c:1516
#33 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387f8f50)
    at shader/slang/slang_emit.c:1516
#34 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x387ff100)
    at shader/slang/slang_emit.c:1516
#35 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x38803ec8)
    at shader/slang/slang_emit.c:1516
#36 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x38808768)
    at shader/slang/slang_emit.c:1516
#37 0x00002b00b6bd2c49 in emit (emitInfo=0x7ffff7c6de20, n=0x388087d8)
    at shader/slang/slang_emit.c:1527
#38 0x00002b00b6bd2c14 in emit (emitInfo=0x7ffff7c6de20, n=0x388088b8)
    at shader/slang/slang_emit.c:1516
#39 0x00002b00b6bd414f in _slang_emit_code (n=0x388088b8, vt=0x3878dbe0, 
    prog=0x33ac0600, withEnd=1 '\001', log=0x7ffff7c702c0)
    at shader/slang/slang_emit.c:1828
#40 0x00002b00b6bcb22a in _slang_codegen_function (A=0x7ffff7c6df50, 
    fun=<value optimized out>) at shader/slang/slang_codegen.c:3127
#41 0x00002b00b6bd0319 in parse_function (C=0x7ffff7c70270, O=0x7ffff7c6e010, 
    definition=1, parsed_func_ret=0x7ffff7c6e048)
    at shader/slang/slang_compile.c:1777
#42 0x00002b00b6bd0b63 in parse_code_unit (C=0x7ffff7c70270, 
    unit=0x7ffff7c6e210, program=<value optimized out>)
    at shader/slang/slang_compile.c:1847
#43 0x00002b00b6bd139e in _slang_compile (ctx=0x24c187c0, shader=0x33f09be0)
    at shader/slang/slang_compile.c:1899
#44 0x00002b00b6b6ee49 in _mesa_compile_shader (ctx=0x24c187c0, 
    shaderObj=<value optimized out>) at shader/shader_api.c:994
#45 0x00000000004790ee in GL_Backend_CompileShader (programobject=1, 
    shadertypeenum=35632, shadertype=0x5b19ca "fragment", numstrings=15, 
    strings=0x7ffff7c78c60) at gl_backend.c:874

In frame 45, I did:

(gdb) print *(char*[15]*)strings
$6 = {0x5b6c27 "#define FRAGMENT_SHADER\n", 0x5b6c40 "\n", 
  0x5b69e8 "#define MODE_LIGHTDIRECTIONMAP_MODELSPACE\n", 0x5b6c40 "\n", 
  0x5b6c40 "\n", 0x5b6c40 "\n", 0x5b6c40 "\n", 0x5b6c40 "\n", 0x5b6c40 "\n", 
  0x5b6b25 "#define USECONTRASTBOOST\n", 0x5b6b4e "#define USESPECULAR\n", 
  0x5b6c40 "\n", 0x5b6c40 "\n", 0x5b6c40 "\n", 
  0x5b3dc8 "// ambient+diffuse+specular+normalmap+attenuation+cubemap+fog shader\n// written by Forest 'LordHavoc' Hale\n\n// common definitions between vertex shader and fragment shader:\n\n#ifdef __GLSL_CG_DATA_TYPE"...}

I will attach an archive with the GLSL shader that DP uses, and a version that Mesa accepts without an abort (mainly by splitting expressions into multiple lines and removing casts that only are necessary on nvidia cards to use half precision floats). There seems to be a problem with the preprocessor too, but I can't figure it out.

BTW, good news: Nexuiz works in Mesa with acceptable frame rates (about 60fps) on 320x240 with r_showsurfaces 1 (textureless flat rendering). At the lowest possible "real" settings, I get 7fps. Good work, now if you could get this up to 15fps... doing a 320x240 r_showsurfaces 1 timedemo (tdem demo1)... resulted in actually "just playable" performance:

1910 frames 56.9229441 seconds 33.5541324 fps, one-second min/avg/max: 21 34 50

Comment 3 Rudolf Polzer 2007-07-06 16:41:42 UTC

Created attachment 10611 [details]
default.glsl file of DarkPlaces and a version that doesn't cause SIGABRT

Comment 4 Brian Paul 2007-07-26 16:09:33 UTC

I just tried the shaders you attached with Mesa CVS and I didn't get a crash.  I ran "progs/demos/fslight -fs default.glsl"

Could you try the latest code from git?

Comment 5 Rudolf Polzer 2007-08-11 06:23:59 UTC

Created attachment 11096 [details]
ready-made fragment and vertex shader program for use in the fs example program

It only crashes if you actually define the features of the shader that crash...

and actually, I can make that shader work in DP now if I add:

#undef MODE_LIGHTDIRECTIONMAP_MODELSPACE
#undef MODE_LIGHTDIRECTION

Known other options that cause a crash are USEFOG and USEOFFSETMAPPING (however, USEOFFSETMAPPING and USEOFFSETMAPPING_RELIEFMAPPING together are fine). The crash about USECONTRASTBOOST is gone in current git.

For the test app, the shader has to be copied to two parts, and one made as vertex and one made as fragment shader. I will now attach two "pre-made" files with MODE_LIGHTDIRECTION enabled so you can get the crash easier...

rpolzer:+ware/libs/mesa-git/src/mesa/progs/demos 25> LD_PRELOAD=../../lib64/libGL.so ./fslight -fs ~/.nexuiz/data/glsl/default.glsl.f -vs ~/.nexuiz/data/glsl/default.glsl.v 
freeglut (./fslight): Unable to create direct context rendering for window './fslight'
This may hurt performance.
fslight: read 11751 bytes from shader file /home/rpolzer/.nexuiz/data/glsl/default.glsl.f
zsh: segmentation fault (core dumped)  LD_PRELOAD=../../lib64/libGL.so ./fslight -fs  -vs 

With the nvidia driver, these shader files compile and work.

I also noticed two other Mesa bugs... I edited the GLSL shader for DarkPlaces to #undef the features that crash (namely, any of the MODEs (so only lightmapped stuff works)) and Nexuiz started up - but I found:

- fog crashes (can be reproduced with the test app as described)
- there seems to be no antialiasing of textures, see http://hagger.rbi.informatik.uni-frankfurt.de/~polzer/nexuiz/temp/mesa_glsl_1.jpg; the non-GLSL path does antialias: http://hagger.rbi.informatik.uni-frankfurt.de/~polzer/nexuiz/temp/mesa_glsl_0.jpg
- weapons and other "colormapped" stuff (which has multiple texture layers on top of each other) appears very wrong, and r_glsl_offsetmapping_reliefmapping 1; r_glsl_offsetmapping 1 causes the world to be tinted in green: http://hagger.rbi.informatik.uni-frankfurt.de/~polzer/nexuiz/temp/mesa-reliefmapping.jpg

The bugs mentioned do not happen on the nvidia driver, even with the GLSL shader file edited to remove features that still crash Mesa (also attached in the tarball).

Comment 6 Peter Cordes 2007-11-11 18:19:04 UTC

This is a problem for the i965 dri driver as well as the software-only mesa.  I just updated my mesa, and now nexuiz doesn't work with glsl shaders enabled.  This is a regression from ~October 20th, last time I updated my mesa from git.  (On AMD64 Ubuntu Gutsy, with libdrm, kernel-side drm, and mesa from latest git as of Nov. 11th).

 This might be due to merging the 965-glsl git branch?  Was that supposed to add hardware-supported shaders, or did mesa already have that?  Does anyone maintain a feature/bugfix changelog for mesa, along the lines of http://kernelnewbies.org/Linux_2_6_23 or a few other Linux kernel news sites?

 Anyway, this bug is now a problem for "normal users".  Nexuiz on g965 is (or was) very playable.  Disabling glsl 2.0 shaders in the "video" menu makes it ok again, but I didn't look carefully, so I don't know what kind of impact that has on frame rates or visual appearance.  It only crashes in-game, not at the menu, and the crash doesn't take down the X server or anything.

 The output is:
nexuiz.bin: shader/slang/slang_emit.c:962: emit_move: Assertion n->Children[0]->Store->Index >= 0' failed.
Aborted (core dumped)

 so it's probably the same bug as Rudolf found.

Comment 7 Brian Paul 2007-11-12 07:53:23 UTC

It would be most helpful if you could trim down the shader to the smallest possible size that still causes the segfault or assertion.  I think the problem is in the compiler's code generator.  I don't have time o disect these big shaders right now.

A high-level list of changes in Mesa is maintained in the docs/relnotes*.html files, BTW.

Comment 8 Mateusz Kaduk 2008-12-24 01:00:58 UTC

Created attachment 21463 [details]
nexuiz-24-12-08.log and default.glsl

Hi,
According to nexuiz log. 
Commands (log_file "file.log" | developer 1 | r_glsl 1 | log_file"")

I get the following
==============================================================
r_glsl 1
from disk... GLSL shader glsl/default.glsl generic diffuse compiled.
from disk... GLSL shader glsl/default.glsl generic compiled.
from disk... program link log:

Too many texture samplers
GLSL shader glsl/default.glsl lightmap fog glow failed!  some features may not work properly.
from disk... program link log:

Too many texture samplers
GLSL shader glsl/default.glsl lightmap fog failed!  some features may not work properly.
from disk... program link log:

Too many texture samplers
GLSL shader glsl/default.glsl lightmap failed!  some features may not work properly.

OpenGL 2.0 shaders disabled - unable to find a working shader permutation fallback on this driver (set r_glsl 1 if you want to try again)
==============================================================

I would like to point out "Too many texture samplers.
Its hardcoded 8 while most of games require more.

/root/GEM/mesa/progs/demos$ ./fslight -fs ~/.nexuiz/data/glsl/default.glsl 
efault.glsl 

Above command works just fine, default.glsl is received in ~/.nexuiz/data/glsl/
after r_glsl_dumpshader typed on game console.

The problem is with higher sampling.

Comment 9 Brian Paul 2008-12-31 07:58:19 UTC

Mesa is currently limited to eight texture units.  This mostly comes from the limit of 8 sets of texture coordinates.  With some work, the limit could be raised.  I'll see what I can do.

Comment 10 Brian Paul 2008-12-31 11:07:22 UTC

OK, I've raised the max samplers to 16 in Mesa.  However, different GPUs will have different limits.  The i965 driver is limited to 8 at this time, but I believe that could be increased.

Comment 11 Corbin Simpson 2008-12-31 13:35:22 UTC

(In reply to comment #10)
> OK, I've raised the max samplers to 16 in Mesa.  However, different GPUs will
> have different limits.  The i965 driver is limited to 8 at this time, but I
> believe that could be increased.
> 

Radeons also have the 8-texture-unit limit, at least up to the X1950. (Haven't figured out yet whether the limit's higher on HD series cards.) Might be worth seeing if it's possible to use less textures, or at least have a fallback shader for older cards...

Comment 12 Mateusz Kaduk 2009-01-01 03:33:28 UTC

(In reply to comment #11)
> > OK, I've raised the max samplers to 16 in Mesa.  However, different GPUs will
> > have different limits.  The i965 driver is limited to 8 at this time, but I
> > believe that could be increased.

After latest commit
mesa: increase max texture image units and GLSL samplers to 16

When I run nexuiz with "GLSL" and "GLSL color handling" I experience a freez.
When I run just "GLSL" without "GLSL color handling" I get the following log

====== Log started (Thu Jan 01 12:26:42 2009) ======
developer 1
r_glsl 1
from disk... GLSL shader glsl/default.glsl generic diffuse compiled.
from disk... GLSL shader glsl/default.glsl generic compiled.
from disk... program link log:
Too many texture samplers (9, max is 8)
GLSL shader glsl/default.glsl lightmap fog failed!  some features may not work properly.
from disk... program link log:
Too many texture samplers (9, max is 8)
GLSL shader glsl/default.glsl lightmap failed!  some features may not work properly.
OpenGL 2.0 shaders disabled - unable to find a working shader permutation fallback on this driver (set r_glsl 1 if you want to try again)
Server can't keep up: 100.0% CPU, 24.00% lost, offset avg 17.0ms, max 100.0ms, sdev 12.9ms
log_file ""
====== Log stopped (Thu Jan 01 12:27:06 2009) ======

Looks like output is more verbose, saying that we are trying to use 9 and max is still 8.

Comment 13 Brian Paul 2009-01-01 13:09:55 UTC

I don't understand your previous comment.  If you tested 'without "GLSL color handling"' why is the shader being compiled?

In any case, I've raised the limit on texture samplers in the i965 driver to 16.  Seems to work with my test program.

I'm not sure which DRI driver you're using.

Comment 14 Mateusz Kaduk 2009-01-02 12:36:06 UTC

After latest commits in nexuiz

====== Log started (Fri Jan 02 21:33:19 2009) ======
log_file "test.txt"
developer 1
r_glsl 1
log_file ""
====== Log stopped (Fri Jan 02 21:33:27 2009) ======

Looks like the problem in Nexuiz with shader compilation failing due to overcoming texture sampling limit is gone, so I am closing this report.
Thanks.

Comment 15 Roland Scheidegger 2009-01-05 07:17:42 UTC

(In reply to comment #11)
> Radeons also have the 8-texture-unit limit, at least up to the X1950. (Haven't
> figured out yet whether the limit's higher on HD series cards.) Might be worth
> seeing if it's possible to use less textures, or at least have a fallback
> shader for older cards...
Not true. All radeons from r300 and up have 16 texture samplers (image units). Texture coord units are indeed limited to 8 but that shouldn't be a problem.
(I don't know what the limits are for r600, apparently sampler id must be from 0-17 so that would be 18 samplers, not sure what the limit is on "coord units" for the r600 if there even is one (might just be limited by available registers or something) but I'd guess it could handle more than 8).

Comment 16 Adam Jackson 2009-08-24 12:27:11 UTC

Mass version move, cvs -> git

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.