Bug 4730 - SProcShmCreatePixmap() fails to byte-swap 'pid'
Summary: SProcShmCreatePixmap() fails to byte-swap 'pid'
Alias: None
Product: xorg
Classification: Unclassified
Component: Server/General (show other bugs)
Version: git
Hardware: All Linux (All)
: high normal
Assignee: Xorg Project Team
QA Contact:
Depends on:
Reported: 2005-10-10 02:00 UTC by Neil Campbell
Modified: 2005-10-20 18:50 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:

Patch to byteswap xShmCreatePixmapReq.pid in SProcShmCreatePixmap(). (616 bytes, patch)
2005-10-10 02:45 UTC, Neil Campbell
no flags Details | Splinter Review

Description Neil Campbell 2005-10-10 02:00:25 UTC
SProcShmCreatePixmap() in file programs/Xserver/Xext/shm.c swaps all the
multi-byte members of the xShmCreatePixmapReq struct except for pid.  If the
application and the X server are running in different endianness, calls to
XShmCreatePixmap will fail.

This is difficult to reproduce unless you happen to have a locally running X
client in a different endianness mode to the server, but if you have and you try
to use XShmCreatePixmap the X server will return an error.

I have a patch which fixes this bug, I will attach it to this report.  The bug
was found in release 6.8.2, but the problem exists in CVS HEAD as well.
Comment 1 Neil Campbell 2005-10-10 02:45:03 UTC
Created attachment 3530 [details] [review]
Patch to byteswap xShmCreatePixmapReq.pid in SProcShmCreatePixmap().

The attached patch corrects the problem I'm seeing with the endian-mismatched
client and server.  In the common case for shared memory (ie. that in which the
server and client are the same endianness) this shouldn't change anything, as
the data should not be byteswapped anyway.
Comment 2 Matthieu Herrb 2005-10-10 03:45:18 UTC
Doesn't this patch break compatibility for old clients connecting to a new server?
Comment 3 Neil Campbell 2005-10-10 03:49:47 UTC
(In reply to comment #2)
> Doesn't this patch break compatibility for old clients connecting to a new server?
I don't believe so, unless the client has been hacked to reverse the pid itself.
 If the endianness of client and server is the same, which it nearly always is,
then the swap won't have any effect.  If the endianness is different, it just
won't work as is; the server won't recognise the pid as a valid id.
Comment 4 Adam Jackson 2005-10-21 11:50:25 UTC
lib/Xext/XShm.c does not byte-swap the pid element of the request (nor should
it), so Neil's patch is correct.  since this change only affects the SProc path
it will not break normal (same-endianness) MIT-SHM clients.

applied to head, will be in 7.0RC2.  thanks!

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.