Bug 6336

Summary: xconsole: On 64-bit linux systems, xconsole can fail with SegFault when using result from ptsname (man page for ptsname is incorrect or misleading)
Product: xorg Reporter: Ferris McCormick <fmccor>
Component: App/otherAssignee: Xorg Project Team <xorg-team>
Status: RESOLVED FIXED QA Contact: Xorg Project Team <xorg-team>
Severity: normal    
Priority: high CC: dberkholz, jeremyhu, joshuabaergen
Version: 7.0.0   
Hardware: x86 (IA32)   
OS: Linux (All)   
Whiteboard: 2011BRB_Reviewed
i915 platform: i915 features:
Attachments:
Description Flags
1.0.2-fix-segfault.patch none

Description Ferris McCormick 2006-03-22 00:33:20 UTC 
On 64-bit userland linux systems (specifically, AMD64), xconsole can fail
because the prototype for ptsname does not get defined, it defaults to (32-bit)
int, and the result gets used as a (64-bit) pointer.  Following little patch
fixes the problem for me, but I can't test it in general:
=================================================
--- xconsole.c- 2006-03-21 13:18:12.000000000 +0000
+++ xconsole.c  2006-03-21 13:19:53.000000000 +0000
@@ -67,6 +67,10 @@
 extern FILE *fdopen(int, char const *);
 #endif
 
+#ifdef __linux__
+extern char *ptsname(int);
+#endif
+
 static void inputReady(XtPointer w, int *source, XtInputId *id);
 static long TextLength(Widget w);
 static void TextReplace(Widget w, int start, int end, XawTextBlock *block);
===============================================

Please see https://bugs.gentoo.org/show_bug.cgi?id=127000 for a long
explanation, especially Comments #6, #7.

Regards,
Ferris
Comment 1 Daniel Stone 2006-03-22 00:43:43 UTC 
man ptsname suggests:
       #define _XOPEN_SOURCE
       #include <stdlib.h>

       char *ptsname(int fd);

is this somehow deficient?
Comment 2 Ferris McCormick 2006-03-22 00:49:35 UTC 
(In reply to comment #1)
> man ptsname suggests:
>        #define _XOPEN_SOURCE
>        #include <stdlib.h>
> 
>        char *ptsname(int fd);
> 
> is this somehow deficient?

Not at all.  I don't see how I could have looked at the man page for so long and
read over that without comprehension.  That looks like the proper fix, of course.

Red faced,
Ferris
Comment 3 Joshua Baergen 2007-01-27 11:38:03 UTC 
Created attachment 8518 [details] [review]
1.0.2-fix-segfault.patch

This fixes it for me.
Comment 4 Alan Coopersmith 2007-01-27 21:16:16 UTC 
Except that always defining _XOPEN_SOURCE will break it on other platforms.
Welcome to the hell that is brought to us by standards that seek to make
portability easier.
Comment 5 Joshua Baergen 2007-02-25 08:21:21 UTC 
(In reply to comment #4)
> Except that always defining _XOPEN_SOURCE will break it on other platforms.
> Welcome to the hell that is brought to us by standards that seek to make
> portability easier.
> 

So is Ferris' solution the better one, then?
Comment 6 Alan Coopersmith 2007-02-25 09:28:45 UTC 
No, #defining _XOPEN_SOURCE for Linux only is preferable, so that you get the
prototype provided by the platform.   A number of the other X modules already
have code in configure.ac to add _XOPEN_SOURCE on Linux only.
Comment 7 Daniel Stone 2007-02-25 09:50:43 UTC 
s/Linux/GNU userland/g
Comment 8 Daniel Stone 2007-02-27 01:31:09 UTC 
Sorry about the phenomenal bug spam, guys.  Adding xorg-team@ to the QA contact so bugs don't get lost in future.
Comment 9 Jeremy Huddleston Sequoia 2011-09-24 21:56:22 UTC 
Is this still an issue?  I don't seen anything as having changed in X11, but maybe glibc is less stupid now.  Please submit an improved patch or I suggest we will probably close this as "NOTOURBUG"
Comment 10 Alan Coopersmith 2011-09-25 09:15:11 UTC 
autoconf provides a macro to set the _*_SOURCE flags now:
AC_USE_SYSTEM_EXTENSIONS

If this is still needed for xconsole, that would be the preferred fix.
Comment 11 Alan Coopersmith 2017-08-31 00:02:20 UTC 
AC_USE_SYSTEM_EXTENSIONS was added in xconsole 1.0.7 - let us know if that
doesn't fix this issue.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.