Can you install the debug packages for the xserver and get a proper backtrace with gdb?  Instructions:
http://wiki.x.org/wiki/Development/Documentation/ServerDebugging

Created attachment 59352 [details]
backtrace

Thanks for looking at this. Backtrace attached. Please let me know if this is not suitable.

Looks like radeon_setup_kernel_mem() is failing silently before allocating info->front_bo. Can you trace its execution to find out why it's failing?

Which version of libdrm do you have installed?

(In reply to comment #3)
> Looks like radeon_setup_kernel_mem() is failing silently before allocating
> info->front_bo. Can you trace its execution to find out why it's failing?
> 
> Which version of libdrm do you have installed?

libdrm 2.4.33.

How would I go about tracing radeon_setup_kernel_mem() ? Sadly I'm not terribly familiar with the gnu toolchain.

Created attachment 59408 [details] [review]
Fail more verbosely and gracefully

(In reply to comment #4)
> How would I go about tracing radeon_setup_kernel_mem() ? Sadly I'm not terribly
> familiar with the gnu toolchain.

See the 'All the gdb commands you'll ever need' section on the referenced wiki page.

Anyway, this patch should make the failure more verbose and graceful. Please provide the log output from starting X with it applied.

Unfortunately it seems running with that patch applied makes the driver behave differently and terminates before I am able to get a backtrace. If I recompile without the patch applied, it behaves as previously described.

Starting program: /usr/bin/Xorg
[Thread debugging using libthread_db enabled]
[tcsetpgrp failed in terminal_inferior: Operation not permitted]
[Inferior 1 (process 28881) exited with code 01]

I've stepped through a bit more (clumsily) without the patch applied. The gdb session with that is attached; perhaps it's helpful.

Created attachment 59625 [details]
stepping through from RADEONScreenInit_KMS()

(In reply to comment #6)
> Unfortunately it seems running with that patch applied makes the driver behave
> differently and terminates before I am able to get a backtrace.

Yes, that's what I was trying to say in comment #5. :) Let me reiterate the request from there:

Please attach the Xorg.0.log from starting X with the patch applied.


(In reply to comment #7)
> I've stepped through a bit more (clumsily) without the patch applied. The gdb
> session with that is attached; perhaps it's helpful.

Unfortunately not:

* As mentioned in comment #3, you'd have to step through radeon_setup_kernel_mem, before the SIGSEGV occurs in RADEONScreenInit_KMS.
* Single stepping in gdb is only really useful when gdb can access the source code files.

My apologies. I actually took some time to see if I could get this working with the fglrx driver - no luck there either. Anyway, Xorg.0.log with patch applied is attached.

Created attachment 60040 [details]
Xorg.0.log with patch applied

Created attachment 60066 [details] [review]
Debug best failure

Can you patch libdrm with that patch and run Xorg from ssh and capture stderr and attach it here. Thx

Created attachment 61025 [details]
Xorg log with Jerome's libdrm error debug messages

Hi,

Different hardware here (AMD E1-1200), PALM Chip 1002:9809.
kernel 3.3.3 and Debian's DDX 6.14.4, libdrm 2.4.33 and mesa 8.0.2.

KMS is working, X server is not. I have the same error when starting the X server. Excerpt with Jerome's patch on libdrm:

radeon_surface_sanity 919
radeon_surface_best 1014
(EE) RADEON(0): radeon_surface_best failed
(EE) RADEON(0): radeon_setup_kernel_mem failed

Full Xorg log in attachment.

Chris,

A patch has been submitted in libdrm to fix this bug:
http://cgit.freedesktop.org/mesa/drm/commit/?id=cf7cc62a9817a495264bbb037f0175cef9bd7a53
It's now in libdrm 2.4.34

It adds the PCI id of your card, so it should work now.
Please re-open the bug if it doesn't.