Bug 2200 - RFE: Solaris should compile Xserver and X11 applications with -xcheck=stkovf
RFE: Solaris should compile Xserver and X11 applications with -xcheck=stkovf
Status: RESOLVED WONTFIX
Product: xorg
Classification: Unclassified
Component: Server/General
git
All Solaris
: high enhancement
Assigned To: Xorg Project Team
http://access1.sun.com/techarticles/s...
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2005-01-02 23:04 UTC by Roland Mainz
Modified: 2005-10-08 02:16 UTC (History)
4 users (show)

See Also:
i915 platform:
i915 features:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Roland Mainz 2005-01-02 23:04:02 UTC
RFE: Solaris should compile Xserver and X11 applications with -xcheck=stkovf to
catch stack overflows (which could otherwise be used to inject code or other
nasty things).
Comment 1 Roland Mainz 2005-01-02 23:04:39 UTC
References:
http://access1.sun.com/techarticles/stack-overflow.html
Comment 2 Seongbae Park 2005-01-03 08:26:12 UTC
Let's clarify a few things first.

-xcheck=stkovf flag protects against a stack overflow but not a buffer overflow.
The difference is that a stack overflow can occur even in a non-buggy program
that simply uses more stack space than the actual stack size, but a buffer
overflow occurs due to a program bug that sometimes can overwrite the
non-current stack frame. So compiling Xserver and X11 applications with
-xcheck=stkovf will help debugging a potential stack overflow problems, but
that's not likely to make them secure from the buffer overflow attack.
The former alone might be valuable enough to trade the performance hit of
-xcheck=stkovf, but I'm not so sure.
Comment 3 Alan Coopersmith 2005-01-03 08:51:14 UTC
The only stack overflows (other than buffer overflows on the stack which 
get fixed other ways) I remember seeing in X servers on Solaris have been 
attempts to allocate way too much memory via alloca() - personally I'd rather
solve those by removing alloca() as its lack of any way to check errors is
not a good design.

We get a limited amount of protection against overflow exploits on SPARC & AMD64 
platforms by linking with mapfiles that mark stack & data segments as
non-executable (see /usr/lib/ld/map.noexstk & on Solaris 10 
/usr/lib/ld/map.noexdata - limited because it doesn't prevent all classes of
exploits, just the easier ones).
Comment 4 ajax at nwnk dot net 2005-01-22 12:48:10 UTC
gcc systems using the ProPolice stack protector require a few extra symbols to
be exported by the module loader, at least for elfloader configurations.  i
don't know how -xcheck=stkovf works, but since similar methods require ABI
extensions this one might too.  just something to be aware of.
Comment 5 Seongbae Park 2005-01-24 08:34:09 UTC
ajax, -xcheck=stkovf is not what you think it is (stack overflow != buffer
overflow on stack). Please read the link Roland provided if you're interested in
more detail. BTW, -xcheck=stkovf does not require any ABI change.
Comment 6 Roland Mainz 2005-03-07 13:16:30 UTC
(In reply to comment #3)
> The only stack overflows (other than buffer overflows on the stack which 
> get fixed other ways) I remember seeing in X servers on Solaris have been 
> attempts to allocate way too much memory via alloca() - personally I'd rather
> solve those by removing alloca() as its lack of any way to check errors is
> not a good design.

OK... should be close this bug as WONTFIX then ?

> We get a limited amount of protection against overflow exploits on SPARC & AMD64 
> platforms by linking with mapfiles that mark stack & data segments as
> non-executable (see /usr/lib/ld/map.noexstk & on Solaris 10 
> /usr/lib/ld/map.noexdata - limited because it doesn't prevent all classes of
> exploits, just the easier ones).

Which compiler switch is needed to get that working in Solaris 10 ?
Comment 7 Alan Coopersmith 2005-03-07 13:54:53 UTC
To use the mapfiles mentioned, the flags would be:
-Wl,-M,/usr/lib/ld/map.noexstk -Wl,-M,/usr/lib/ld/map.noexdata

The Xorg sun.cf already sets the first flag on Solaris 9 and later - see the
NonExecStackLoadFlag setting.   I've never gotten around to adding the second
for Solaris 10 - it would have to be x86 only since it will break the SPARC PLT
as noted in the comments in the file.   (When building the binaries distributed
with Solaris we actually use custom mapfiles which contain the commands noted
in the above, plus those from the /usr/lib/ld/map.pagealign and on SPARC
the non-executable bss flag described in the comments in the map.noexdata.)

For autoconf use, the STSF configure.ac shows an example of one way to detect
and use these:
case "$target" in
        *-*-solaris*)
                AC_CHECK_FILE([/usr/lib/ld/map.noexstk],[LDFLAGS="${LDFLAGS}
-Wl,-M/usr/lib/ld/map.noexstk"])
                AC_CHECK_FILE([/usr/lib/ld/map.pagealign],[LDFLAGS="${LDFLAGS}
-Wl,-M/usr/lib/ld/map.pagealign" ; SHAREDLIB_LDFLAGS="${SHAREDLIB_LDFLAGS}
-Wl,-M/usr/lib/ld/map.pagealign"])
                ;;
esac
Comment 8 Alan Coopersmith 2005-10-08 19:16:49 UTC
I checked in changes to the sun.cf Imake config file in CVS on 8/23/2005 to 
use the noexstk & noexdata mapfiles where possible.

Since the -xcheck=stkovf flag should mainly be used by adding to the flags when
debugging and not for general use, I'm closing this bug as WONTFIX as Roland
suggested a while ago.