Bug 17191 - libdrm has sign-extension bug in drmIoctl
Summary: libdrm has sign-extension bug in drmIoctl
Status: RESOLVED FIXED
Alias: None
Product: DRI
Classification: Unclassified
Component: libdrm (show other bugs)
Version: XOrg git
Hardware: x86-64 (AMD64) FreeBSD
: medium normal
Assignee: Default DRI bug account
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-08-18 12:27 UTC by Coleman Kane
Modified: 2008-08-18 14:11 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments
Patch to libdrm to fix the problem on FreeBSD/amd64 (360 bytes, patch)
2008-08-18 12:27 UTC, Coleman Kane
no flags Details | Splinter Review

Description Coleman Kane 2008-08-18 12:27:03 UTC
Created attachment 18361 [details] [review]
Patch to libdrm to fix the problem on FreeBSD/amd64

FreeBSD amd64 reports a sign-extension on the ioctl calls which pass through drmIoctl in xf86drm.c. The problem is that the request is typed as "int" (a signed, 32-bit type), while the ioctl system call expects this to be an "unsigned long" (an unsigned, 64-bit type). As it first performs the signed conversion, then the field extension, any ioctl's with the high-bit set will have all high 32-bits set in the ioctl passed to the driver. The drm driver is only looking for unsigned long ioctl requests, so none of these high ioctls will succeed. This eventually leads to a hang in the X-server on my system.

For instance: ioctl request 0xc0001432 becomes 0xffffffffc0001432, but the drm driver is looking for 0x00000000c0001432, so the subsequent ioctls never succeed.
Comment 1 Robert Noland 2008-08-18 14:11:23 UTC
Committed, thanks.

robert.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.