Bug 17191

Summary: libdrm has sign-extension bug in drmIoctl
Product: DRI Reporter: Coleman Kane <ckane>
Component: libdrmAssignee: Default DRI bug account <dri-devel>
Status: RESOLVED FIXED QA Contact:
Severity: normal    
Priority: medium    
Version: XOrg git   
Hardware: x86-64 (AMD64)   
OS: FreeBSD   
Whiteboard:
i915 platform: i915 features:
Attachments:
Description Flags
Patch to libdrm to fix the problem on FreeBSD/amd64 none

Description Coleman Kane 2008-08-18 12:27:03 UTC
Created attachment 18361 [details] [review]
Patch to libdrm to fix the problem on FreeBSD/amd64

FreeBSD amd64 reports a sign-extension on the ioctl calls which pass through drmIoctl in xf86drm.c. The problem is that the request is typed as "int" (a signed, 32-bit type), while the ioctl system call expects this to be an "unsigned long" (an unsigned, 64-bit type). As it first performs the signed conversion, then the field extension, any ioctl's with the high-bit set will have all high 32-bits set in the ioctl passed to the driver. The drm driver is only looking for unsigned long ioctl requests, so none of these high ioctls will succeed. This eventually leads to a hang in the X-server on my system.

For instance: ioctl request 0xc0001432 becomes 0xffffffffc0001432, but the drm driver is looking for 0x00000000c0001432, so the subsequent ioctls never succeed.
Comment 1 Robert Noland 2008-08-18 14:11:23 UTC
Committed, thanks.

robert.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.