Bug 97938 - Rowstride integer overflow, a wrong idiom
Summary: Rowstride integer overflow, a wrong idiom
Status: RESOLVED MOVED
Alias: None
Product: pixman
Classification: Unclassified
Component: pixman (show other bugs)
Version: git master
Hardware: x86-64 (AMD64) All
: medium normal
Assignee: Oded Gabbay
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-09-26 17:38 UTC by Alessandro Vesely
Modified: 2018-06-05 15:27 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments
patch of pixman-fast-path.c (1.91 KB, patch)
2016-09-26 17:38 UTC, Alessandro Vesely
Details | Splinter Review

Description Alessandro Vesely 2016-09-26 17:38:15 UTC
Created attachment 126795 [details] [review]
patch of pixman-fast-path.c

it is wrong to compute offsets like so:

   int rowstride = something;
   char *buffer = base_ptr + y*rowstride + x*4;

That idiom fails in 64-bit architectures where integers are 32 bit.  Consider a not-so-uncommon A0 poster at 600 dpi.  It results in a 19860x28080 image.  While width and heights are 16 bit numbers, their product multiplied by a bpp of 4 results in a negative integer.

A better choice than int would be ptrdiff_t (POSIX ssize_t wouldn't work on x86-16, according to
http://stackoverflow.com/questions/8649018/what-is-the-difference-between-ssize-t-and-ptrdiff-t)

The patch attached wildly uses (long).  I'd suggest to define a one-liner inline function using a well thought cast, which can always be tweaked for specific architectures, should problems arise.

Ale
Comment 1 Siarhei Siamashka 2016-09-27 23:15:18 UTC
Thanks! Yes, this seems to be a pretty much obvious problem. Too bad that it can be easily overlooked when programming in C. Please send your patch to the pixman mailing list with the appropriate commit message and a Signed-off-by tag. Extending the test suite to demonstrate this bug (and similar bugs of the same class) would be very much welcome too.

You can find more information here - https://cgit.freedesktop.org/pixman/plain/README
Comment 2 GitLab Migration User 2018-06-05 15:27:38 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/pixman/pixman/issues/14.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.