Bug 73473 - Potential crash bug in src/gallium/auxiliary/rtasm/rtasm_execmem.c
Potential crash bug in src/gallium/auxiliary/rtasm/rtasm_execmem.c
Status: RESOLVED FIXED
Product: Mesa
Classification: Unclassified
Component: Drivers/DRI/nouveau
unspecified
x86-64 (AMD64) Linux (All)
: medium critical
Assigned To: Nouveau Project
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-01-10 13:47 UTC by Jaak Ristioja
Modified: 2014-01-27 13:31 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments
gallium/rtasm: add support for SELinux (2.02 KB, patch)
2014-01-10 18:28 UTC, Emil Velikov
Details | Splinter Review
patch adding check for PaX mprotect (1.59 KB, patch)
2014-01-17 19:21 UTC, Amadeusz
Details | Splinter Review

Note You need to log in before you can comment on or make changes to this bug.
Description Jaak Ristioja 2014-01-10 13:47:01 UTC
glxgears[4186]: segfault at ffffffffffffffff ip 000078805fc4b901 sp 00007ce9598e21c0 error 7 in nouveau_dri.so[78805f7d1000+136c000]

Stracing it revealed that the crash happens after a mmap(NULL, 10485760, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE,MAP_ANONYMOUS, -1, 0) syscall returns -1.

I think it might be caused by the return value of mmap not being checked in src/gallium/auxiliary/rtasm/rtasm_execmem.c, leading to the the memory being accessed somewhere else.

So it probably needs some

  if (exec_mem == MAP_FAILED)

check somewhere.

PS: Sorry if this is not the correct component.
Comment 1 Emil Velikov 2014-01-10 14:11:52 UTC
Hi Jaak

If you're getting consistent crashes in glxgears I would recommend using gdb to get a backtrace of the problem.

With that said, I suspect that the problems is elsewhere for a few reasons
* There are more than a handful cases when mmap fails and I have yet to notice after 3+ years constant use of nouveau any glxgears segfaults.
* I would suspect other mesa users will be affected and this problem would be well know/resolved by now.

Apart from the backtrace would you mind attaching your dmesg output after the problem/segfault ?
Can you reproduce with the swrast driver ?
$ LIBGL_ALWAYS_SOFTWARE=1 glxgears

Cheers
Comment 2 Jaak Ristioja 2014-01-10 15:38:53 UTC
(In reply to comment #1)
> If you're getting consistent crashes in glxgears I would recommend using gdb
> to get a backtrace of the problem.

I wish I could but gdb only shows ?? in backtrace:

(gdb) thread apply all bt full

Thread 1 (process 2782):
#0  0x00006a0a172d4901 in ?? ()
No symbol table info available.
#1  0x0000000000000000 in ?? ()
No symbol table info available.

> With that said, I suspect that the problems is elsewhere for a few reasons
> * There are more than a handful cases when mmap fails and I have yet to
> notice after 3+ years constant use of nouveau any glxgears segfaults.
> * I would suspect other mesa users will be affected and this problem would
> be well know/resolved by now.
> 
> Apart from the backtrace would you mind attaching your dmesg output after
> the problem/segfault ?

[17407.732321] grsec: From 5.4.2.83: denied RWX mmap of <anonymous mapping> by /usr/bin/glxgears[glxgears:2866] uid/euid:1000/1000 gid/egid:100/100, parent /bin/bash[bash:2860] uid/euid:1000/1000 gid/egid:100/100
[17407.732328] glxgears[2866]: segfault at ffffffffffffffff ip 0000685b13c99901 sp 000077a968f66e50 error 7 in nouveau_dri.so[685b1381f000+136c000]
[17407.732342] grsec: From 5.4.2.83: Segmentation fault occurred at ffffffffffffffff in /usr/bin/glxgears[glxgears:2866] uid/euid:1000/1000 gid/egid:100/100, parent /bin/bash[bash:2860] uid/euid:1000/1000 gid/egid:100/100
[17407.732356] grsec: From 5.4.2.83: denied resource overstep by requesting 4096 for RLIMIT_CORE against limit 0 for /usr/bin/glxgears[glxgears:2866] uid/euid:1000/1000 gid/egid:100/100, parent /bin/bash[bash:2860] uid/euid:1000/1000 gid/egid:100/100

So it appears to try to read memory at ffffffffffffffff (i.e. -1 alias MAP_FAILED). It doesn't matter why mmap fails (EPERM). What matters is that it is documented that it may fail, but errors do not appear to be handled in code.

I don't think Gentoo should be expected to mark every single application using OpenGL to be allowed mmap RWX memory. I use Intel + OpenGL on my Gentoo Hardened laptop with no problems.

> Can you reproduce with the swrast driver ?
> $ LIBGL_ALWAYS_SOFTWARE=1 glxgears

$ DISPLAY=:0.0 LIBGL_ALWAYS_SOFTWARE=1 glxgears; echo $?
LLVM ERROR: Allocation failed when allocating new memory in the JIT
Can't allocate RWX Memory: Operation not permitted
1

I'm using Hardened Gentoo (kernel is vanilla-3.12.6 + genpatches-3.12-7 + grsecurity-3.0-3.12.6-201401021726; gcc --version is "gcc (Gentoo Hardened 4.7.3-r1 p1.4, pie-0.5.5) 4.7.3").
Comment 3 Emil Velikov 2014-01-10 18:28:15 UTC
Created attachment 91834 [details] [review]
gallium/rtasm: add support for SELinux

You're absolutely right here. Seems like there aren't many hardened/selinux users of the gallium drivers.

The attached patch should do the job, can you give it a try ?
Comment 4 Jaak Ristioja 2014-01-11 10:29:20 UTC
(In reply to comment #3)
> Created attachment 91834 [details] [review] [review]
> gallium/rtasm: add support for SELinux
> 
> You're absolutely right here. Seems like there aren't many hardened/selinux
> users of the gallium drivers.
> 
> The attached patch should do the job, can you give it a try ?

I tried the patch on Mesa 5.1.9 under Gentoo.

patching file src/gallium/auxiliary/rtasm/rtasm_execmem.c
Hunk #1 succeeded at 60 (offset -1 lines).
Hunk #2 succeeded at 72 (offset -1 lines).
Hunk #3 succeeded at 90 (offset -1 lines).
Hunk #4 succeeded at 103 (offset -1 lines).
Hunk #5 succeeded at 115 (offset -1 lines).

Running headless over SSH I could not verify today that the glxgears were actually shown in X, but here are the results I got:

$ DISPLAY=:0.0 glxgears
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
29168 frames in 5.0 seconds = 5833.411 FPS
29463 frames in 5.0 seconds = 5892.528 FPS
...

This still gives the kernel message of RWX mmap being denied which is normal.

$ DISPLAY=:0.0 LIBGL_ALWAYS_SOFTWARE=1 glxgears
LLVM ERROR: Allocation failed when allocating new memory in the JIT
Can't allocate RWX Memory: Operation not permitted

This also gives the kernel message, but exits immediately after with exit status 1.
Comment 5 Jaak Ristioja 2014-01-11 12:22:53 UTC
PS: I'm not using selinux. So I guess I'm unable to test that part of the patch.
Comment 6 Emil Velikov 2014-01-11 13:12:08 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > Created attachment 91834 [details] [review] [review] [review]
> > gallium/rtasm: add support for SELinux
> > 
> > You're absolutely right here. Seems like there aren't many hardened/selinux
> > users of the gallium drivers.
> > 
> > The attached patch should do the job, can you give it a try ?
> 
> I tried the patch on Mesa 5.1.9 under Gentoo.
> 
I hope you mean 9.1.5 here :)

> patching file src/gallium/auxiliary/rtasm/rtasm_execmem.c
> Hunk #1 succeeded at 60 (offset -1 lines).
> Hunk #2 succeeded at 72 (offset -1 lines).
> Hunk #3 succeeded at 90 (offset -1 lines).
> Hunk #4 succeeded at 103 (offset -1 lines).
> Hunk #5 succeeded at 115 (offset -1 lines).
> 
> Running headless over SSH I could not verify today that the glxgears were
> actually shown in X, but here are the results I got:
> 
> $ DISPLAY=:0.0 glxgears
> Running synchronized to the vertical refresh.  The framerate should be
> approximately the same as the monitor refresh rate.
> 29168 frames in 5.0 seconds = 5833.411 FPS
> 29463 frames in 5.0 seconds = 5892.528 FPS
> ...
> 
> This still gives the kernel message of RWX mmap being denied which is normal.
> 
Great, thanks.

> $ DISPLAY=:0.0 LIBGL_ALWAYS_SOFTWARE=1 glxgears
> LLVM ERROR: Allocation failed when allocating new memory in the JIT
> Can't allocate RWX Memory: Operation not permitted
> 
> This also gives the kernel message, but exits immediately after with exit
> status 1.
I'm guessing that some work may be needed for LLVM.

(In reply to comment #5)
> PS: I'm not using selinux. So I guess I'm unable to test that part of the
> patch.
I thought that was the case. Either way big thanks for the report.
Comment 7 Jaak Ristioja 2014-01-11 13:45:14 UTC
(In reply to comment #4)
> Running headless over SSH I could not verify today that the glxgears were
> actually shown in X, but here are the results I got:

Confirmed over SSH and VNC that glxgears works properly.
Comment 8 Amadeusz 2014-01-17 19:21:04 UTC
Created attachment 92300 [details] [review]
patch adding check for PaX mprotect

As I said on #gentoo-hardened channel, I wouldn't like to see the SELinux part of this patch to be merged. Provided SELinux check effectively requires allowing all applications to be allowed access to write|exec memory regardless of if it is needed or not.

I tested patch without the SELinux part and it worked fine on my PaX & SELinux enabled system. Starting glxgears didn't bring down whole X server as was the case before applying patch ;) .


If one wants to avoid "grsec: denied RWX mmap" messages probably something along the lines of patch I attached (based on checks from the SELinux one and http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-libs/libffi/files/libffi-3.0.13-emutramp_pax_proc.patch?revision=1.2&view=markup) would be needed.
Comment 9 Jaak Ristioja 2014-01-17 23:01:55 UTC
Personally, I'd keep both the selinux and PaX parts out of the patch.

I suggest simply checking the return value for the mmap call. I think this would suffice. Keep it simple. There are also other reasons why mmap can fail (e.g. due to ulimit) and this should be checked. Please fix this first, add PaX/selinux/whatever support later. :)

PS: While in the selinux patch the selinux code can be conditionally #ifdef enabled, in the PaX patch the PaX code is not, meaning this results in a minor performance impact for all users.
Comment 10 Emil Velikov 2014-01-27 13:24:49 UTC
Yay the crash should happen no more

commit 4dd445f1cf80292f10eda53665cefc2a674d838d
Author: Emil Velikov <emil.l.velikov@gmail.com>
Date:   Fri Jan 10 18:00:17 2014 +0000

    gallium/rtasm: handle mmap failures appropriately

    For a variety of reasons mmap (selinux and pax to name
    a few) and can fail and with current code. This will
    result in a crash in the driver, if not worse.

    This has been the case since the inception of the
    gallium copy of rtasm.

    Cc: 9.1 9.2 10.0 <mesa-stable@lists.freedesktop.org>
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=73473
Comment 11 Emil Velikov 2014-01-27 13:31:57 UTC
Jaak Ristioja

Considering how long it took for people to review this patch it may be worth opening another bug about
"[gallium] Add selinux/pax support to auxiliary/rtasm" and select "Mesa core" as the component. This will hopefully get some more people looking in this direction.

Closing this bug as the crash should be resolved.