Bug 32277 - overflow in calculate_miptree_layout_r300
Summary: overflow in calculate_miptree_layout_r300
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Drivers/DRI/R600 (show other bugs)
Version: git
Hardware: Other All
: medium major
Assignee: Default DRI bug account
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-12-09 14:10 UTC by Alban Browaeys
Modified: 2010-12-20 07:06 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments
Fix for the levels array overflow (1.30 KB, patch)
2010-12-09 14:10 UTC, Alban Browaeys
Details | Splinter Review
backtrace of the calloc failure in radeon drm due to this previous corruption (3.67 KB, text/plain)
2010-12-09 14:11 UTC, Alban Browaeys
Details
valgrind output from such an etracer run which shows the overflow (41.82 KB, text/plain)
2010-12-09 14:14 UTC, Alban Browaeys
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alban Browaeys 2010-12-09 14:10:12 UTC
Created attachment 40964 [details] [review]
Fix for the levels array overflow

extremetuxracer 0.5 beta2 shows up a bug in r600c. 

The levels array in radeon_mipmap_tree structure is defined with a size of RADEON_MIPTREE_MAX_TEXTURE (ie 13). Though in radeon_try_alloc_miptree the size of numLevels can overflow this size.
Then in calculate_miptree_layout_r300 the loop write out of the array.
Which leads to calloc failure in bo_open from radeon_gem_bo due to corrupted memory.

This patch fixes this by setting the numLevels max to RADEON_MIPTREE_MAX_TEXTURE.
Comment 1 Alban Browaeys 2010-12-09 14:11:16 UTC
Created attachment 40965 [details]
backtrace of the calloc failure in radeon drm due to this previous corruption
Comment 2 Alban Browaeys 2010-12-09 14:14:53 UTC
Created attachment 40967 [details]
valgrind output from such an etracer run which shows the overflow

the overflow happens in  in radeon_mipmap_tree.c:195 ie calculate_miptree_layout_r300 and has origin radeon_try_alloc_miptree .
Comment 4 Alban Browaeys 2010-12-20 07:06:40 UTC
User reported the issue as fixed. Can be closed. Thank you


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.