Bug 10529 - Incorrect lighting/clipping in Neverwinter Nights
Summary: Incorrect lighting/clipping in Neverwinter Nights
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Drivers/DRI/i915 (show other bugs)
Version: 6.5
Hardware: x86 (IA32) Linux (All)
: medium normal
Assignee: Default DRI bug account
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2007-04-04 18:51 UTC by Roderick B. Greening
Modified: 2007-07-09 01:16 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
Image showing problem in NWN (215.09 KB, image/jpeg)
2007-04-04 18:53 UTC, Roderick B. Greening
Details
Latest Mesa GIT test - Fog incorrectly rendered solid (165.62 KB, image/jpeg)
2007-04-17 06:30 UTC, Roderick B. Greening
Details
Correctly rendered scene based on Nov 2, 206 GIT snapshot (187.58 KB, image/jpeg)
2007-04-17 06:33 UTC, Roderick B. Greening
Details
Screenshot of Google Earth with fog (500.62 KB, image/jpeg)
2007-04-22 06:35 UTC, Jan Larres
Details
Fog disapear from Warcraft III Frozen Throne opening screen (970.44 KB, image/png)
2007-07-09 00:24 UTC, Adrianus B. Kurnadi
Details
Warcraft III opening screen in ubuntu edgy (mesa 6.5.1) (842.25 KB, image/png)
2007-07-09 00:32 UTC, Adrianus B. Kurnadi
Details

Description Roderick B. Greening 2007-04-04 18:51:33 UTC
Eversince xorg-server 1.2.0 and Mesa 6.5.2 (both requirements of xorg-x11 7.2), I have a strange issue with Neverwinter Nights (native Linux client version - patched to latest).

The issue is hard to explain, but I will try. If someone wants a screenshot I can forward. Basically, it looks like the gamma correction is off (everything has a bright glow). However, if I zoom in (camera changes) then the image begins to appear correct. It's almost as if the 3D engine isn't properly setting/reading the correct coordinates for lighting and camera or something.

As well, the field of view for objects in the distance seems to be clipped too close, rather than being in the distance. 

To verify this is not related to xf86-video-intel driver or xorg-server 1.2.0, I have tried various versions of both and no change. However, if I keep latest intel driver, latest xorg-server and downgrade mesa to 6.5.1, the problem goes away (when it loads the i915_dri.so from Mesa 6.5.1). 

I have tried the latest GIT versions of Mesa, xorg-server, x11-drm, intel agp for kernel, and the intel video driver. No luck in correcting the problem.

Also, when using all GIT versions, with i915tex_dri.so enabled, Neverwinter Nights loads, but none of the text appears in the main menu, and some of the grapics/images do no show or are partially loaded/cut off.

Can anyone help determine what is broken here? Something changed from 6.5.1 -> 6.5.(2|3).
Comment 1 Roderick B. Greening 2007-04-04 18:53:55 UTC
Created attachment 9479 [details]
Image showing problem in NWN
Comment 2 Roland Scheidegger 2007-04-05 04:31:56 UTC
It would help greatly if you could isolate the change which caused the broken behaviour between mesa 6.5.1 and 6.5.2 with git-bisect.
Comment 3 Roderick B. Greening 2007-04-05 13:35:07 UTC
Ok, I can git it a try, though the version I used for 6.5.1 was prepackaged on Gentoo. I tried several other pre-packaged versions there and no change, which is what led me to try a GIT version.

Any suggestions on how to proceed (never performed a bisect before)? I suspect it's in the i915 code, but not entirely sure. 
Comment 4 Roland Scheidegger 2007-04-06 04:55:16 UTC
(In reply to comment #3)
> Ok, I can git it a try, though the version I used for 6.5.1 was prepackaged on
> Gentoo. I tried several other pre-packaged versions there and no change, which
> is what led me to try a GIT version.
> 
> Any suggestions on how to proceed (never performed a bisect before)? I suspect
> it's in the i915 code, but not entirely sure. 
It doesn't really matter where the bug was.
If you know 6.5.1 works and 6.5.2 doesn't, you'd do:
git bisect start
git bisect good mesa_6_5_1 (you could use any sha1 id here too)
git bisect bad mesa_6_5_2
then do a make and test
depending if it worked or not, type git bisect good or git bisect bad
repeat the last two steps until git spits out the broken revision.
It can happen the compile fails because of partial rebuild, try a make realclean in that case. If the compile really fails or you can't test the bug you want because of other broken things, git bisect can deal with that too. See the git-bisect man page.

Comment 5 Roderick B. Greening 2007-04-13 08:48:45 UTC
Not sure how I can go about this without breaking things completely.

The GIT version under Gentoo is using a live ebuild (i.e. I am not directly installing from GIT, but rather the ebuild is pulling down GIT sources and building it from there and installing to proper locations).

If I were to try and do this by hand on my production system, I'm not sure what to do to install to correct locations without breaking other things.

Is there some other detail/test I can provide/try first? If not, I'll try the manual GIT build/install.
Comment 6 Roland Scheidegger 2007-04-13 09:33:20 UTC
(In reply to comment #5)
> If I were to try and do this by hand on my production system, I'm not sure what
> to do to install to correct locations without breaking other things.
> 
> Is there some other detail/test I can provide/try first? If not, I'll try the
> manual GIT build/install.

You don't need to install the dri driver. You can set LIBGL_DRIVERS_PATH env var to mesa/lib and libGL will pick it up from there. In case the libGL is incompatible with the new driver (there were some changes some time ago, can't remember when exactly) I guess you can force loading the right libGL with LD_LIBRARY_PATH or LD_PRELOAD. Or you just copy those two files over, but these two should really be enough.
Comment 7 Roderick B. Greening 2007-04-13 09:55:55 UTC
Ok. Will try it out.

Thanks.
Comment 8 Roderick B. Greening 2007-04-13 11:10:31 UTC
Ok,

Testing with following command:

export LD_LIBRARY_PATH="/usr/local/src/mesa/lib/" 
export LIBGL_DRIVERS_PATH="/usr/local/src/mesa/lib" 
nwn

I'm hoping this is enough to correctly test. The first bisect works using the LEGACY3D option for the intel driver and the above commands. I'll keep bisecting this way to see if I can generate a break.

Once I find the break, what is the next step? Generate some patch to revert changes to latest GIT revision? Or is there a built-in mechanism I can use? Sorry for the questions.

Cheers, and thanks for the help.
Comment 9 Roderick B. Greening 2007-04-13 11:57:33 UTC
Got down to this commit:

author  Brian Paul <brian.paul@tungstengraphics.com>
   Thu, 2 Nov 2006 22:39:29 +0000 (22:39 +0000)
  committer  Brian Paul <brian.paul@tungstengraphics.com>
   Thu, 2 Nov 2006 22:39:29 +0000 (22:39 +0000)
  commit  e812081253b2857b59d17f40dc6c9909e1957d87
  tree  4ba16ceab8be20aa839cde0a2fb3eb66eacd4599
  parent  d78f65cd6cf04dafcf50b4014de9d28546badcfb

Then I had problems compiling with a problem with the ATI stencil commit. Tried make realclean... couldn't get past it. It one of the commits +/- 2 around this one that caused my problem.

Still seeing what I can do to narrow it down.
Comment 10 Roderick B. Greening 2007-04-13 12:03:15 UTC
Ok, I took a guess and went git-bisect good when I had the ATI compile problem... (this may not have been 100% correct). Assuming it was, it reported this as possible bad commit:

prometheus mesa # git-bisect bad
af864bb0daf3aba9bc2526d207925eea0c44d162 is first bad commit
commit af864bb0daf3aba9bc2526d207925eea0c44d162
Author: Brian Paul <brian.paul@tungstengraphics.com>
Date:   Thu Nov 2 22:40:43 2006 +0000

    check for shader extensions for OpenGL 2.0

:040000 040000 5050892225d10b60505fc9eb1095b98d9d3be2a6 36bd3195d3a14e26228cc56ffa79fc37d4cbfa71 M      src

I'm going to see if I can step back to the ATI break and choose git-bisect bad rather than good and see what happens... just to be sure.
Comment 11 Roderick B. Greening 2007-04-13 12:27:22 UTC
I definitely guessed wrong. There's no way the commit indicated below was bad.

So, that means it has to be:

Fix fragment program fog problems found with Glean.
author  Brian Paul <brian.paul@tungstengraphics.com>
   Thu, 2 Nov 2006 21:43:06 +0000 (21:43 +0000)
  committer  Brian Paul <brian.paul@tungstengraphics.com>
   Thu, 2 Nov 2006 21:43:06 +0000 (21:43 +0000)
  commit  d78f65cd6cf04dafcf50b4014de9d28546badcfb
  tree  9ef84f1d00410c71ca63aa99f4d2cf61a20f6e3d
  parent  67a101759ec0b84e4f30fbb329e86ac5031cf7e7

Which turned on fog... yep. Broken for me.

So, what can we do now that I have isolated the problem? Should I generate a patch to reverse this and apply to Mesa 6.5.2 or GIT head to verify my problem disappears or will someone take a stab at attempting to find out why this is broken. 

It was previously disabled for some reason, and then it got re-enabled via this patch.
Comment 12 Roderick B. Greening 2007-04-13 16:20:11 UTC
I re-did the entire test again just to be 100% sure...

prometheus mesa # git bisect bad
d78f65cd6cf04dafcf50b4014de9d28546badcfb is first bad commit
commit d78f65cd6cf04dafcf50b4014de9d28546badcfb
Author: Brian Paul <brian.paul@tungstengraphics.com>
Date:   Thu Nov 2 21:43:06 2006 +0000

    Fix fragment program fog problems found with Glean.
    1. add PROGRAM_CONSTANT to switch() in src_vector().
    2. use _mesa_append_fog_code() to handle fog options in i915ProgramStringNotify().
    3. Re-enable some vertex attribute emit code that was previously disabled in i915ValidateFragmentProgram().

:040000 040000 c0316509ddb831826ffff6b7529c2d10b6ef0023 f0d24ed57b611405a8a3a8f279c416727fe8ec00 M      src

This is specifically the point at which it all broke for me.

I really hope someone can help with this now that I have pinpointed it. If you need me to test some more or any patches, let me know.
Comment 13 Roland Scheidegger 2007-04-13 18:34:51 UTC
(In reply to comment #12)
>     2. use _mesa_append_fog_code() to handle fog options in
> i915ProgramStringNotify().
>     3. Re-enable some vertex attribute emit code that was previously disabled
> in i915ValidateFragmentProgram().
Sounds reasonable that these could cause the trouble you're seeing. Fog setup seems a little weird, that re-enabled attribute emit code means fog is now emitted twice (separate as float4 and as specular alpha), not sure if the hardware likes that (especially together with spec color). I'm not familiar with the driver though and lack the hardware, so someone else will have to look at that.
Comment 14 Roderick B. Greening 2007-04-13 18:48:25 UTC
Any chance we can get one of the Intel/Tungsten committers to investigate?

Also, if I comment out the "_mesa_append_fog_code" call, it reverts to previous behaviour (i.e. games runs fine again). Obviously, this is not the desired result, as that prevents the fragment code for fog from being added (which appears to be broken. I guess the problem likely lies in either the call itself and some missing check, or in the implementation of _mesa_append_fog_code() included in programopt.c

Here's what I did just to temporarily get around my problem. Not for general use, but points to the issue I have:

--- src/mesa/drivers/dri/i915/i915_fragprog.c
+++ src/mesa/drivers/dri/i915/i915_fragprog.c
@@ -940,7 +940,6 @@ static void i915ProgramStringNotify( GLc

       if (p->FragProg.FogOption) {
          /* add extra instructions to do fog, then turn off FogOption field */
-         _mesa_append_fog_code(ctx, &p->FragProg);
          p->FragProg.FogOption = GL_NONE;
       }
    }
Comment 15 Roderick B. Greening 2007-04-13 21:27:20 UTC
After some additional digging, the clipping I am seeing is actually an artifact of incorrectly rendered fog. 

I believe the commit I mentioned before along with the later patch from bug 9686 both contribute to my issue.

When I disable the _mesa_append_fog_code() call, the lighting appears normal, but I still get the incorrectly rendered fog/haze in the distance (this appears as a solid fill rendered rather than the semi transparant it should be and therefore looks like the screen is clipped). I believe that the solution to bug 9686 is either incomplete or incorrect when coupled with the implementation of _mesa_append_fog_code().

I hope all of this is useful...

Comment 16 Brian Paul 2007-04-16 14:17:15 UTC
I found one mistake in the _mesa_append_fog_code() function.  If fog mode == GL_LINEAR we need to clamp the fog blend factor when it's computed.

I don't know what fog mode is used though (I don't have the game).
Comment 17 Roderick B. Greening 2007-04-17 06:02:07 UTC
Is there a way I can find out through some debugging techniques to assist here?

Also, do you have a fix (committed or otherwise) to the problem you found? I can patch/build/test pretty quickly/easily (assuming this game uses the GL_LINEAR).
Comment 18 Roderick B. Greening 2007-04-17 06:30:51 UTC
Created attachment 9635 [details]
Latest Mesa GIT test - Fog incorrectly rendered solid

So, latest GIT version corrects the lighting problems I was originally experiencing. 

It does not, however correct the incorrectly rendered fog which appears to be a solid wall.

Attached picture shows the incorrectly rendered fog.
Comment 19 Roderick B. Greening 2007-04-17 06:33:14 UTC
Created attachment 9636 [details]
Correctly rendered scene based on Nov 2, 206 GIT snapshot

Here is the way the scene should look with correctly rendered fog. The last time this worked correctly was Nov 2, 2006 (based on the git bisect previously preformed).
Comment 20 Roderick B. Greening 2007-04-17 06:35:27 UTC
Comment on attachment 9479 [details]
Image showing problem in NWN

Lighting corrected with latest GIT. So, this snapshot is obsolete. Fog rendering remains an issue (see other image attachments).
Comment 21 Roland Scheidegger 2007-04-17 07:18:36 UTC
(In reply to comment #18)
> Created an attachment (id=9635) [details]
> Latest Mesa GIT test - Fog incorrectly rendered solid
> 
> So, latest GIT version corrects the lighting problems I was originally
> experiencing. 
Brian corrected the clamping issue. And yes, nwn uses indeed linear fog.

> It does not, however correct the incorrectly rendered fog which appears to be a
> solid wall.
> 
> Attached picture shows the incorrectly rendered fog.
Looks slightly unnatural :-).
The fog code in this driver looks like it could need some cleanup, it seems to have degenerated a bit. There are for instance references like this:
      /* Hack: make sure fog is correctly enabled according to this
       * fragment program's fog options.
       */
except if you look at the function call it's a no-op nowadays...
As far as I can tell the whole i915_update_fog function is basically obsolete since the appending of the fog in the fragment program. Not sure though, hw fog might still get enabled if this is called before the update of the fragment program (which would add a bogus fog calc in addition of the correct one probably). So, could you try never setting S5_FOG_ENABLE in i915_state.c? You can just comment it out.
Comment 22 Keith Whitwell 2007-04-17 07:21:30 UTC
Some of this rewrite has already occurred on the index-swtnl branch.  Haven't noticed that there's been a major reworking of the i915tex driver on that branch?  That's because git commit reports only work for small changes - the big ones get flushed down the drain & nobody notices them.
Comment 23 Roderick B. Greening 2007-04-17 09:09:45 UTC
(In reply to comment #21)
> Looks slightly unnatural :-).
> The fog code in this driver looks like it could need some cleanup, it seems to
> have degenerated a bit. There are for instance references like this:
>       /* Hack: make sure fog is correctly enabled according to this
>        * fragment program's fog options.
>        */
> except if you look at the function call it's a no-op nowadays...
> As far as I can tell the whole i915_update_fog function is basically obsolete
> since the appending of the fog in the fragment program. Not sure though, hw fog
> might still get enabled if this is called before the update of the fragment
> program (which would add a bogus fog calc in addition of the correct one
> probably). So, could you try never setting S5_FOG_ENABLE in i915_state.c? You
> can just comment it out.
> 

I commented out the entire if/then/else block containing the S5_FOG_ENABLE (I assume that's what you wanted), and re-built. The only change in doing this is that the screen appears slightly brighter. The solid wall is still there instead of the properly rendered fog.

Did you instead want me to comment out only one of those lines from the block?
Comment 24 Roderick B. Greening 2007-04-17 09:16:38 UTC
(In reply to comment #22)
> Some of this rewrite has already occurred on the index-swtnl branch.  Haven't
> noticed that there's been a major reworking of the i915tex driver on that
> branch?  That's because git commit reports only work for small changes - the
> big ones get flushed down the drain & nobody notices them.
> 

So, are there changes in this branch that need to get pushed up to master now?

Comment 25 Keith Whitwell 2007-04-17 09:17:49 UTC
Not yet ready - it is a big set of changes.
Comment 26 Roland Scheidegger 2007-04-17 17:37:21 UTC
(In reply to comment #23)
> I commented out the entire if/then/else block containing the S5_FOG_ENABLE (I
> assume that's what you wanted), and re-built. The only change in doing this is
> that the screen appears slightly brighter. The solid wall is still there
> instead of the properly rendered fog.
> 
> Did you instead want me to comment out only one of those lines from the block?
No, that should be sufficient. Must be some other problem then.
Comment 27 Roderick B. Greening 2007-04-18 05:34:30 UTC
Keep the suggestions coming. I really do want to narrow this down and help fix it.
Comment 28 Roderick B. Greening 2007-04-18 09:53:07 UTC
Looks like 6.5.3 is being prepped... any chance we can try and fix this problem prior?
Comment 29 Brian Paul 2007-04-18 10:17:03 UTC
If I back out the changes so things are as they were on Nov. 2, progs/demos/fogcoord breaks, as does the glean test.

I don't know what's special/different about NWN.  Is there a free demo of NWN available?

I'm interested in fixing this problem, but I'm not going to delay 6.5.3 because of it.
Comment 30 Roland Scheidegger 2007-04-18 17:45:48 UTC
(In reply to comment #29)
> If I back out the changes so things are as they were on Nov. 2,
> progs/demos/fogcoord breaks, as does the glean test.
> 
> I don't know what's special/different about NWN.  Is there a free demo of NWN
> available?
There is a demo, but only for windows (and it's old, though I don't think the graphic engine changed much).

I don't think there's really anything special about what nwn does, no vertex / fragment progs, from some quick gdb observation fog is linear, fog source fragment_depth, no color sum neither, and start 30, end 45, density 0.2 or 0.1 in a particular scene.
Comment 31 Roderick B. Greening 2007-04-20 06:11:15 UTC
(In reply to comment #29)
> If I back out the changes so things are as they were on Nov. 2,
> progs/demos/fogcoord breaks, as does the glean test.
> 
> I don't know what's special/different about NWN.  Is there a free demo of NWN
> available?
> 
> I'm interested in fixing this problem, but I'm not going to delay 6.5.3 because
> of it.
> 
http://nwn.bioware.com/downloads/demo.html

Not sure if the Linux ver works with the demo or not.

Basically, you can get NWN at any bargain bin for <10$ (at least the original with no expansion). THe Diamond edition, was <20$ and included the 2 expansions and other modules.

If there are tests you want me to perform, I will. Just let me know.
Comment 32 Jan Larres 2007-04-22 06:32:28 UTC
In addition to NWN, I also have a problem with Google Earth, which is probably related: The more I zoom in, the more the scene gets covered with fog, until at some point you can only see the blue-whiteish fog and nothing else. As Google Earth is freely available, this may be a better help with debugging.
Comment 33 Jan Larres 2007-04-22 06:35:02 UTC
Created attachment 9686 [details]
Screenshot of Google Earth with fog
Comment 34 Roland Scheidegger 2007-05-21 18:34:19 UTC
(In reply to comment #31)
> If there are tests you want me to perform, I will. Just let me know.
Could you try again with git head?
Comment 35 Roderick B. Greening 2007-05-22 07:34:43 UTC
I sure will try again today. Were there some specific fixes applied that may be related? Just curious as I follow the commits looking for interesting Intel related updates..

Thanks.
Comment 36 Roland Scheidegger 2007-05-22 07:43:00 UTC
(In reply to comment #35)
> I sure will try again today. Were there some specific fixes applied that may be
> related? Just curious as I follow the commits looking for interesting Intel
> related updates..
Sure the fog fixes - I wouldn't have asked otherwise. They weren't exactly i915 specific, but it might have only affected those drivers.
Comment 37 Roderick B. Greening 2007-05-23 05:56:25 UTC
I'm so happy. Rebuilt using latest GIT from yesterday. Problem is now gone, which means it wasn't an issue with NWN (so I can stop asking them for help). 

:)

Thanks so much.
Comment 38 Adrianus B. Kurnadi 2007-07-09 00:24:56 UTC
Created attachment 10624 [details]
Fog disapear from Warcraft III Frozen Throne opening screen

I find the changes from mesa 6.5.1 to 6.5.2 causing changes in Frozen Throne opening screen. In 6.5.2 some of the water is not visible and the sky becomes dark. In 6.5.1 it shows a foggy scenery, in 6.5.3 and 7.0.0 the fog is missing. (shown in the attachment)
Comment 39 Adrianus B. Kurnadi 2007-07-09 00:32:04 UTC
Created attachment 10625 [details]
Warcraft III opening screen in ubuntu edgy (mesa 6.5.1)

The attachment shows the opening screen of Frozen Throne using opengl Mesa 6.5.1. This is looks most similar to Frozen Throne using d3d.
Comment 40 Roland Scheidegger 2007-07-09 01:16:47 UTC
(In reply to comment #39)
> Created an attachment (id=10625) [details]
> Warcraft III opening screen in ubuntu edgy (mesa 6.5.1)
> 
> The attachment shows the opening screen of Frozen Throne using opengl Mesa
> 6.5.1. This is looks most similar to Frozen Throne using d3d.
Looks like the same bug to me. Try a newer mesa version (7.0 or git).


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.