Bug 18599

Summary: Executing display lists containing smooth multi-color shading with translucency causes assertion failure if color is vertex delivered before glBegin block
Product: Mesa Reporter: Michael Saunders <r.michael.saunders>
Component: Mesa coreAssignee: mesa-dev
Status: RESOLVED MOVED QA Contact:
Severity: normal    
Priority: medium    
Version: git   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:
Attachments: Pseudo code from our OpenGL log showing code that fails and code that doesn't fail.

Description Michael Saunders 2008-11-18 08:23:09 UTC
Created attachment 20420 [details]
Pseudo code from our OpenGL log showing code that fails and code that doesn't fail.

Overview:
---------
Our product uses OpenGL but attempts to buffer the glBegin statements until we are sure that a normal/vertex will be delivered. We do this because to prevent creating empty begin/end blocks because some of our polygons are collapsed but it isn't know at the time the block is first opened. We cache the glBegin request and then When the first normal/vertex is to be delivered we open the block and deliver the first normal/vertex. This introduced an interesting problem with multi-color translucent smooth shading. Because we buffer the glBegin the color we set on the first vertex occurs outside the block instead of inside. In other words we effectively send instructions as follows:
glBegin
glColor4fv
glNormal3d
glVertex3d
glColor4fv
glNormal3d
glVertex3d
...
glEnd

but because of our buffering of glBegin we actually deliver the following to OpenGL:
glColor4fv
glBegin
glNormal3d
glVertex3d
glColor4fv
glNormal3d
glVertex3d
...
glEnd

Note the swapped glBegin with the glColor4fv at the beginning of the delivered instructions. This shouldn't matter because OpenGL is a state engine and in fact produces the correct output if display lists are not used however if we build display lists and then execute them Mesa asserts on line 391 of tnl/t_vertex.c: assert(a[j].inputstride == vptr->stride); I'm not sure if the assertion is correct or not. Unfortunately/fortunately (depending on how you look at it) the default compilation for Mesa's release build has assertions enabled so that GLX servers using Mesa instead of native OpenGL drivers will assert with our product unless we supply our own build of the Mesa libraries with assertions disabled. We can work around the problem by making sure color gets delivered before beginning the block but I thought I should point out the problem if it is indeed a problem.

Steps to reproduce:
-------------------
Attached is a file that contains a log of "pseudo" code (from a dump of our OpenGL instruction log) that is the smallest example I could build. The example had to have one four node polygon with all the colors having the same value and the two other four node polygons having gradient colors The direction of the gradient seemed to matter as well (the gradient in my case varies laterally across the cell width instead of longitudinally across the shared cell edge. The first half of the attachment our applications OpenGL log from the pseudo code that asserts and the second half is the output from the pseudo code that does not assert. The only difference between the two sets is the order of the glColor4fv and the opening glBegin. 

Sorry that the code probably doesn't compile its just an approximation of the code we send to OpenGL. Look for the first glColor4fv call. Note that in the first "BAD" block the glColor4fv call comes before the glBegin(GL_POLYGON) whereas in the second example it comes after. The first case causes Mesa to assert and the second does not.

Actual Results:
---------------
Assertion failure, tnl/t_vertex.c:391, assert(a[j].inputstride == vptr->stride);

Expected Results:
-----------------
Since OpenGL is a state engine the delivery of the glColor4fv before the glBegin shouldn't have made any difference and indeed when display lists are not used it doesn't seem to make any visual difference. Also when display lists are used but assertions disabled it doesn't seem to make any visual difference.

Build Date & Platform:
----------------------
The bug report version label above did not give me an option to enter 7.2 so I put CVS as the version number. This occurs at least on Linux 64 and 32 bit platforms running Mesa 7.2 and Mesa 5.0.
Comment 1 Adam Jackson 2009-08-24 12:31:06 UTC
Mass version move, cvs -> git
Comment 2 Jerome Glisse 2009-08-28 04:26:07 UTC
Does it still happen with more recent mesa ?
Comment 3 Michael Saunders 2009-08-28 06:45:41 UTC
We are still using Mesa 7.2 and we have re-ordered the delivery of verticies to work better with Mesa so I can't say if it still fails or not.
Comment 4 mycae 2010-04-23 12:39:13 UTC
I managed to trigger this bug whilst writing some GL code. Fixing my code worked mind.

Running Debian:
$ aptitude show libgl1-mesa-dev
Package: libgl1-mesa-dev
State: installed
Automatically installed: no
Version: 7.7-4
Priority: optional
Section: libdevel
Maintainer: Debian X Strike Force <debian-x@lists.debian.org>
Uncompressed Size: 69.6k
Depends: libc6-dev, mesa-common-dev (= 7.7-4), libgl1-mesa-glx (= 7.7-4)
Conflicts: libgl-dev, libgl1-mesa-dri-dev
Replaces: libgl-dev, libgl1-mesa-dri-dev
Provides: libgl-dev, libgl1-mesa-dri-dev
Comment 5 GitLab Migration User 2019-09-18 20:22:01 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/mesa/mesa/issues/958.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.