Bug 97938

Summary: Rowstride integer overflow, a wrong idiom
Product: pixman Reporter: Alessandro Vesely <ale.whop>
Component: pixmanAssignee: Oded Gabbay <oded.gabbay>
Status: RESOLVED MOVED QA Contact:
Severity: normal    
Priority: medium CC: siarhei.siamashka
Version: git master   
Hardware: x86-64 (AMD64)   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: patch of pixman-fast-path.c

Description Alessandro Vesely 2016-09-26 17:38:15 UTC
Created attachment 126795 [details] [review]
patch of pixman-fast-path.c

it is wrong to compute offsets like so:

   int rowstride = something;
   char *buffer = base_ptr + y*rowstride + x*4;

That idiom fails in 64-bit architectures where integers are 32 bit.  Consider a not-so-uncommon A0 poster at 600 dpi.  It results in a 19860x28080 image.  While width and heights are 16 bit numbers, their product multiplied by a bpp of 4 results in a negative integer.

A better choice than int would be ptrdiff_t (POSIX ssize_t wouldn't work on x86-16, according to
http://stackoverflow.com/questions/8649018/what-is-the-difference-between-ssize-t-and-ptrdiff-t)

The patch attached wildly uses (long).  I'd suggest to define a one-liner inline function using a well thought cast, which can always be tweaked for specific architectures, should problems arise.

Ale
Comment 1 Siarhei Siamashka 2016-09-27 23:15:18 UTC
Thanks! Yes, this seems to be a pretty much obvious problem. Too bad that it can be easily overlooked when programming in C. Please send your patch to the pixman mailing list with the appropriate commit message and a Signed-off-by tag. Extending the test suite to demonstrate this bug (and similar bugs of the same class) would be very much welcome too.

You can find more information here - https://cgit.freedesktop.org/pixman/plain/README
Comment 2 GitLab Migration User 2018-06-05 15:27:38 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/pixman/pixman/issues/14.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.