This is due to the glxext trying to mmap part of the card's memory. With my card it's trying to allocate half the card's memory (64KiB).

It is possible to map an amount of memory equal to that which the DRM has allocated for itself in its own channel, so my guess is that it's affecting it somehow.

By eliminating the call for the DRM to get its own channel, this mapping succeeds.

I hit this too, all DRM, DDX and DRI from today.

There seems to be a collision of mappings (as noticed by cmn) in the DRM:

[ 3024.352588] [drm:drm_addmap_core] offset = 0xf0000000, size = 0x04000000, type = 0
...
[ 3024.368286] [drm:nouveau_dma_channel_init] 
[ 3024.368291] [drm:drm_addmap_core] offset = 0xf0000000, size = 0x00008000, type = 0
[ 3024.368295] [drm:drm_addmap_core] Matching maps of type 0 with mismatched sizes, (32768 vs 67108864)
[ 3024.368299] [drm] allocated 0x0
[ 3024.368303] [drm] Allocating FIFO number 0
[ 3024.368306] [drm:nouveau_notifier_init_channel] Allocating notifier block in 262
[ 3024.368310] [drm:drm_addmap_core] offset = 0x00000000, size = 0x00001000, type = 3
[ 3024.368314] [drm:drm_addmap_core] AGP offset = 0xe8000000, size = 0x00001000
[ 3024.368317] [drm] allocated 0x0

Notice how at the address 0xf0000000, which is the start of my VRAM, first is allocated the FB block of 64MB, and then a block of 32kB, which overwrites the previous 64MB mapping. I.e., creating the FIFO for the DRM messes things up, but perhaps just triggers a bug elsewhere, in DRI parameter communication.

Later in the log:
[ 3024.370650] [drm:drm_addmap_core] offset = 0xf0008000, size = 0x04000000, type = 0

Now, the DDX seems to be ok with this, because /proc/<Xserver>/maps contains the line:
2b95ab8ed000-2b95af8ed000 rw-s f0008000 00:0b 41226                      /dev/dri/card0

But however, glxinfo tries to do the following syscall:
mmap(NULL, 67108864, PROT_READ|PROT_WRITE, MAP_SHARED, 4, 0xf0000000) = -1 EINVAL (Invalid argument)
which leads to the "drmMap of framebuffer failed" error message.

cmn therefore inferred that libGL does not get the correct information for mapping the framebuffer, but tries it with a wrong address. A quote:
< cmn> glxext knows the offset at which it should request the FB memory from XF86DRIGetDeviceInfo(), but I haven't been able to see where we tell it the address

xf86-video-nouveau/src/nv_dri.c has a comment showing that we expect (and need) the FB to be at the beginning of the physical VRAM space.

This has broken now because FIFOs are created in VRAM, so when DRM allocates a FIFO for itself, it allocates it at the beginning of VRAM, creating a mapping which is much smaller (32KiB) than the one glxext expects to find at that offset (namely, half the VRAM).

A second mapping is created, but currently neither the DDX nor glxext are aware of this change and expect the FB address not to move.

Created attachment 11070 [details] [review]
Move the FB to the high half of VRAM

In the attached patch I move the FB to the higher half of the VRAM. We probably won't run into this bug again, as it's unlikely that that much memory is taken.
In the final solution I'd like for the DDX to know where the FB should be located based on other allocations, but I don't know if that is possible.

Patch committed upstream as a work-around until this mapping disappears.

I have this with lastest Nouveau too.

All aplication with 3D showing this error (glx*,openarena)

PIII 1Ghz, 512MB RAM, nVidia 7600 GS 256MB RAM DDR3[NV4B]

Debian testing/unstable

After lastest update libdrm and xserver-xorg-video-nouveau from Debian experimental repositary, working.

Mass version move, cvs -> git