Bug 20340 - Fuzzy textures when drawing 1:1 with GL_LINEAR
Summary: Fuzzy textures when drawing 1:1 with GL_LINEAR
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Drivers/DRI/r300 (show other bugs)
Version: 7.2
Hardware: Other All
: medium normal
Assignee: Default DRI bug account
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-02-26 15:34 UTC by Owen Taylor
Modified: 2009-10-05 07:42 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
Test case (2.57 KB, text/plain)
2009-02-26 15:34 UTC, Owen Taylor
Details
Screenshot of problem (344 bytes, image/png)
2009-02-26 15:36 UTC, Owen Taylor
Details
Test program for investigation (5.84 KB, text/plain)
2009-05-08 15:55 UTC, Owen Taylor
Details
Remove subpixel offset from viewport (1.93 KB, patch)
2009-05-08 16:01 UTC, Owen Taylor
Details | Splinter Review
fix fuzzy texture for r200 (1.48 KB, patch)
2009-10-04 14:16 UTC, Frederic Crozat
Details | Splinter Review

Description Owen Taylor 2009-02-26 15:34:38 UTC
Created attachment 23344 [details]
Test case

With mesa-libGL-7.2-0.15.fc10.i386 and a rv350, drawing a texture with a filter of GL_LINEAR in such a way that the texels should map 1:1 onto screen pixels seems to result in a small offset.

I'll attach a test case (compilation instructions at top), and a screenshot of the results. The output should be a black and white checkerboard, but if you blow it up, you'll see gray at the edges.
Comment 1 Owen Taylor 2009-02-26 15:36:20 UTC
Created attachment 23345 [details]
Screenshot of problem
Comment 2 Owen Taylor 2009-05-08 15:45:13 UTC
After much investigation, I managed to get a pretty good understanding of what's going on with this test case.

There are basically two sources of fuzziness. The first is a strange leftover in Mesa. In r300_state.c, the viewport is offset by 1/8th pixel:

 /*
  * To correctly position primitives:
  */
 #define SUBPIXEL_X 0.125
 #define SUBPIXEL_Y 0.125

Which comes originally from the r100 driver before the start of version control in 2003. So, I'm not really sure the exact intent of it.

(This 1/8th pixel offset becomes an effective 1/6th pixel offset because of the 1/12th subpixel precision that the driver uses; which is why the observed values from my test program are about 1/6th off.)

Setting this offset to 0 makes things much better. However, depending on the size of the rendered primitives and the texture, there is still slight fuzzing at some (but not all) the boundaries where values that should be 0xff are 0xfb and values that should be 0x00 are 0x04... in other words, we're still 1/64th off for some pixels. 

I analyzed this by writing a fragment shader that compared the incoming texture coordinates and the sampled textured values to the expected value, and found four things that combine to cause the problem:

1) When the rasterizer outputs interpolated texture coordinates, there is some small error in them = ~1/1024th of pixel (for 1:1 pixels and texels). This can be positive or negative.

2) When we sample a texture with a texture coordinate, the sampling is done at quantized positions of 1/64th of a texel.

3) The quantized sample position is determined by truncation - so if the texel coordinate is 3.4999, it samples at 3+31/64th. (!)

4) There's also considerable error additional error when determining sample positions for non-power-of-two rectangular textures; it looks like there might be a multiply and divide with a low-precision fixed point intermediate.

My basic conclusion is that given the above, this additional error probably should just be ignored and considered "good enough". In the end, GL doesn't really make any guarantee that LINEAR filtering  of a 1:1 texture gives perfect result, and if you want that you probably should switch to NEAREST at 1:1. Fixing the gross fixing the gross problem will mean that there won't be sudden popping from fuzzy to not when you switch to NEAREST.

I did experiment some with using the TX_OFFSET field of RS_INST_COUNT; this adds an offset to the texture coordinates coming out of the rasterizer. Since our problem is too small values being truncated, adding a positive offset, often does makes things better. (It helps if the offset is larger than the errors coming out of the rasterizer, but not so larger that it bumps us to the next subtexel position by itself.) This helping is rather coincidental, since the size of the offset isn't a fixed texel offset (what we'd like here), but rather dependent on the the size of the rendered primitive in some fairly complex way. 

I don't think it makes sense to set TX_OFFSET as a resolution to this problem; it's really designed to remove visible artifacts with NEAREST sampling and perhaps also principally for DirectX, where the sample position rules make problems with NEAREST more likely. (It might make sense to set TX_OFFSET independently for other reasons; following whatever fglrx does probably is appropriate.)

Adding the desired 1/128th texel offset directly to texture coordinates would also conceivably be possible when generating the fragment shader, but the complexity is unlikely to be worth the small improvement.

Neither TX_OFFSET nor a manual addition of 1/128th texel really helps the case of NPOT rectangular textures since the errors are larger and introduced right before sampling; and while NPOT rectangular textures may seem like a fringe case, the reason I started investigating this was window textures in a compositing window manager, which are typically NPOT rectangular textures...

So, in conclusion I think the right thing to do is to remove the subpixel offset, and accept the remaining small error. I'll attach:

 A) A patch to do that
 B) The fragment shader test program I used to investigate

I'd like to have a set of piglit runs before/after to make sure that there weren't regressions related to the original intent of "correctly position primitive", but I was unable to get piglit to work for me. (With KMS, there are too many visuals, takes forever, and fails a lot. Without KMS + and with the radeon-rewrite branch of mesa, it crashes my X server.)
Comment 3 Owen Taylor 2009-05-08 15:55:09 UTC
Created attachment 25653 [details]
Test program for investigation
Comment 4 Owen Taylor 2009-05-08 16:01:17 UTC
Created attachment 25655 [details] [review]
Remove subpixel offset from viewport
Comment 5 Alex Deucher 2009-05-13 15:58:45 UTC
committed:
038f0bf5916df5bae1145d234589e5fd528bb7fa

Thanks!
Comment 6 Frederic Crozat 2009-10-04 14:15:26 UTC
reopening, bug is also present in r200 driver, see https://bugzilla.gnome.org/show_bug.cgi?id=597343

I'm attaching r200 version of Owen patch which fixes the issue for me.
Comment 7 Frederic Crozat 2009-10-04 14:16:36 UTC
Created attachment 30043 [details] [review]
fix fuzzy texture for r200
Comment 8 Alex Deucher 2009-10-04 14:59:40 UTC
thanks pushed:
3856c3cc46813ad96ae6f02dec19460193d986ac
Comment 9 Frederic Crozat 2009-10-05 06:58:15 UTC
it was only pushed in 7.6 branch, not in master. Not sure it is wanted or not ?
Comment 10 Alex Deucher 2009-10-05 07:42:43 UTC
(In reply to comment #9)
> it was only pushed in 7.6 branch, not in master. Not sure it is wanted or not ?
> 

7.6 gets merged into master on a regular basis.  So bug fixes go into 7.6, new development happens on master.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.