If Xorg leaked 6 gb of memory for all users, it would have been fixed already.
That it's not fixed is not a sign that no one cares about Xorg memory usage,
but that there's some bug you're hitting that the developers haven't seen before.
Any information you could provide about what's unique in your environment or
output from a leak checker such as valgrind would help us find it.

I think that many "normal" users are not seeing 6GB of memory usage because many of them don't have that much RAM available, but that's besides the point.  Here are the stupidly simple ways able to get Xorg 1.6.1 to leak memory in a repeatable fashion:

1. On my Intel x4500 based laptop using KDE 4.2, the newest 2.6.29 kernel and 2.7.1 Intel drivers, with or without compositing turned on, all I have to do is run a graphical program, then kill that program.  I chose the "kcalc" utility since it was simple to run repeatedly.  The following python program will cause a nice, monotonic increase in memory usage that had ballooned to about 2.6GB after 24 hours:
    
from subprocess import Popen
import time

RUN_CMD = "/usr//bin/kcalc"

# test program that automatically executes, then kills simple x programs.
# Designed to show that x.org will memory leak simply by running & then
# killing trivial programs.

while (True): # continue until the user kills us
  process_list = []

  # launch then kill & wait 10 at a time:
  for II in xrange (0, 10):
     process_list.append (Popen (("kcalc%d" % II), executable=RUN_CMD,
                                 close_fds=True))

  time.sleep (10)  # give programs time to open fully, adjust for your machine

  for nextProc in process_list:
     nextProc.kill ()
     nextProc.wait () # no zombies!


This is a nice, simple leak... I can have 2 windows open and still have X using over 50% of system memory on a 4GB laptop.

2. On my desktop that is using an 8800GT and the 185.18.14 proprietary drivers, the memory handling is... BETTER than the "pure" open-source Intel drivers, so the above method will not cause a leak (or at least leak too slowly to notice). However, the X server still cannot handle memory, and once it gets it's grimy manacles onto a memory page, that page will NOT ever be given back to the system.  
   Here's how I got my desktop's X server to consume 2.9GB of RAM that it will not release in about 5 minutes:
   1. Dump a couple of hundred pdf files into a directory.  They don't have to be huge, mine were just collected at random from my home directory.
   2. Run the following script to open them all up at once:
      #!/bin/sh

      pdfList=`ls *.pdf`

      for nextPdf in ${pdfList}
      do
        okular ${nextPdf} &
      done;

   3. Wait a few minutes to let as many of them open until X actually refuses to spawn new windows.
   4. Do a simple "pkill okular" that kills off all the processes.

   5. Now: X was using 2.9GB of memory with all those windows open... and I'm not complaining about it USING memory when there are over 200 windows open.  I AM complaining about it releasing ZERO resources back to the system after ALL those windows close.  Before you try to tell me this is "caching" or some type of "feature" let me tell you that X is in no way shape or form more responsive using 2.9GB of RAM than it is when I first spawn it only using 36MB, and additionally, if this is a "feature" I should be able to CONFIGURE AND CONTROL the maximum amount of "cache" that X will use.  Since we both know this is not the case, we both know this is a memory leak.  By the way, the 6GB of usage I had before was NOT from opening hundreds of windows at once, it was actually from leaked memory that occurred when I was running programs via WINE for several hours, but I knew that if I said I was using WINE you'd ignore the report and shift blame elsewhere even though WINE had exited 100% but X still used 6GB of memory.

   A few more things to prevent you from shifting blame away:  After all of the tests I describe above, I killed & restarted both kwin and plasma, so their memory was completely freed and then reinitialized.  This did exactly nothing to lower X's memory usage so don't try to say they are the real culprits.

  I'll stop posting these bugs when you personally run all of these tests on commodity, 64 bit hardware using any major graphics chip and current driver from Intel, Nvidia or ATI, and can show me that the X server is using the same amount of memory before the test as after the test.  If you want to blame KDE, I'm warning you that I've seen X behave badly on GNOME desktops too, so run a comprehensive test suite.  I've put up concrete numbers and very simple ways to hammer the X server, so I hope you can tell me how to stop this from happening.

Just confirming, during normal use (firefox, gmusicbrowser, keepassx open, KDE 4, ubuntu 9.04 amd64. Closed firefox, waited 2 hours, mem usage of X stayed the same. Reopened to post this.)

Top:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2903 root      20   0 1124m 297m 4928 S   11 15.0 365:06.42 Xorg

xrestop - Display: :0.0
          Monitoring 37 clients. XErrors: 0
          Pixmaps:   76603K total, Other:     186K total, All:   76789K total


297MB is rather more than 77MB. By 220MB.

Later, will try using Xmonad. I think we can be confident that that will ensure it is not simply a KDE bug of some sort. 

It leaks with XMonad, but not as much: I used the PDF script foxcm provided.

Before:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
21290 root      20   0  385m  22m  11m S    1  1.1   0:01.43 Xorg

After:
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
21290 root      20   0  502m 116m 4504 S    1  5.9   2:17.23 Xorg

xrestop - Display: :0
          Monitoring 14 clients. XErrors: 357
          Pixmaps:    6871K total, Other:      22K total, All:    6893K total

That indicates that at least part of the problem (though not all) comes from KDE.

OK, I've done even more tests to isolate where this leak is coming from:
  1. I tested using fluxbox to eliminate as many possible leaking parties as possible.
  2. I tested GTK and Qt based applications, each one using a program that had as few dependencies as I could find.
  3. I disabled GLX and used the open source NV driver so if the leaks here are driver related, it's technically open source (although I get leaks using either proprietary or open source drivers, and in some ways NV is worse that the proprietary driver).

  Once again, the test is simple: Open up an X program with many instances, then close all the instances completely and see what X.org decides to do with memory.  I'm using Xorg 1.6.1.901 and the 2.1.13 release of the NV driver.

  Xorg's original resident memory when I did a fresh restart (from a direct startx, not using kdm/xdm/gdm) is 35MB.  
   
  I opened up to the maximum allowed instances of gvim, which uses GTK2, but is not otherwise dependant upon GNOME desktop components: Result: X maxed out at 450 MB resident.. BUT it dropped back to 65 MB after all gvim windows were closed, which I am not going to complain about.
 
  The next test ran Qt 4.5.2 based Scribus.  Scribus is a Qt based application, but it is not really dependant upon KDE so none of the KDE components or even libraries are being loaded, just Qt.  In this case, 128 instances (to keep from running out of memory) pushed X.org to 1.4GB of resident usage.... where it stayed after all the scribus processes were shut down.

  So does this mean the problem is fully diagnosed?  I actually think there could be more than one leak I'm seeing, but this one is at least reproducible.  However, this really should be fixed in the X server.  The Qt libraries may be misbehaving, but that is no excuse for X to not release memory when the programs that reserved the memory come to an end.  If a kernel leaked memory whenever a user process didn't clean up perfectly, we'd call it Windows 3.1. The same principle applies here, the X server has the responsibility to clean up memory after a user application terminates, even if the user application is not operating perfectly.

Here's the result of xrestop while X.org is using 1.4GB of RAM:
0 - Fluxbox ( PID:  ?   ):
        res_base      : oxc00000
        res_mask      : ox1fffff
        windows       : 184
        GCs           : 21
        fonts         : 1
        pixmaps       : 182
        pictures      : 4
        glyphsets     : 5
        colormaps     : 1
        passive grabs : 293
        cursors       : 19
        unknowns      : 13
        pixmap bytes  : 6629830
        other bytes   : ~13984
        total bytes   : ~6643814
1 - <unknown> ( PID:  ?   ):
        res_base      : ox0
        res_mask      : ox1fffff
        windows       : 1
        GCs           : 0
        fonts         : 2
        pixmaps       : 0
        pictures      : 0
        glyphsets     : 0
        colormaps     : 0
        passive grabs : 0
        cursors       : 1
        unknowns      : 45
        pixmap bytes  : 4608000
        other bytes   : ~3176
        total bytes   : ~4611176
2 - <unknown> ( PID:  ?   ):
        res_base      : oxe00000
        res_mask      : ox1fffff
        windows       : 0
        GCs           : 0
        fonts         : 0
        pixmaps       : 1
        pictures      : 0
        glyphsets     : 0
        colormaps     : 1
        passive grabs : 0
        cursors       : 0
        unknowns      : 0
        pixmap bytes  : 4608000
        other bytes   : ~24
        total bytes   : ~4608024
3 - Bug 22011 – X really does hog memory - Shiretoko ( PID: 8471 ):
        res_base      : ox1000000
        res_mask      : ox1fffff
        windows       : 321
        GCs           : 173
        fonts         : 1
        pixmaps       : 107
        pictures      : 164
        glyphsets     : 44
        colormaps     : 0
        passive grabs : 0
        cursors       : 7
        unknowns      : 42
        pixmap bytes  : 538243
        other bytes   : ~19048
        total bytes   : ~557291
4 - login ( PID: 6103 ):
        res_base      : oxa00000
        res_mask      : ox1fffff
        windows       : 3
        GCs           : 34
        fonts         : 8
        pixmaps       : 3
        pictures      : 0
        glyphsets     : 0
        colormaps     : 1
        passive grabs : 12
        cursors       : 9
        unknowns      : 2
        pixmap bytes  : 15
        other bytes   : ~9656
        total bytes   : ~9671
5 - xterm ( PID: 6107 ):
        res_base      : ox800000
        res_mask      : ox1fffff
        windows       : 2
        GCs           : 20
        fonts         : 6
        pixmaps       : 1
        pictures      : 0
        glyphsets     : 0
        colormaps     : 1
        passive grabs : 12
        cursors       : 8
        unknowns      : 2
        pixmap bytes  : 0
        other bytes   : ~7224
        total bytes   : ~7224
6 - xterm ( PID: 8466 ):
        res_base      : ox400000
        res_mask      : ox1fffff
        windows       : 2
        GCs           : 12
        fonts         : 6
        pixmaps       : 1
        pictures      : 0
        glyphsets     : 0
        colormaps     : 1
        passive grabs : 12
        cursors       : 8
        unknowns      : 2
        pixmap bytes  : 0
        other bytes   : ~7032
        total bytes   : ~7032
7 - xterm ( PID: 6106 ):
        res_base      : ox600000
        res_mask      : ox1fffff
        windows       : 2
        GCs           : 9
        fonts         : 6
        pixmaps       : 1
        pictures      : 0
        glyphsets     : 0
        colormaps     : 1
        passive grabs : 12
        cursors       : 8
        unknowns      : 2
        pixmap bytes  : 0
        other bytes   : ~6960
        total bytes   : ~6960
8 - xrestop ( PID:  ?   ):
        res_base      : ox1600000
        res_mask      : ox1fffff
        windows       : 1
        GCs           : 1
        fonts         : 0
        pixmaps       : 0
        pictures      : 0
        glyphsets     : 0
        colormaps     : 0
        passive grabs : 0
        cursors       : 0
        unknowns      : 0
        pixmap bytes  : 0
        other bytes   : ~48
        total bytes   : ~48
9 - <unknown> ( PID:  ?   ):
        res_base      : ox1200000
        res_mask      : ox1fffff
        windows       : 1
        GCs           : 1
        fonts         : 0
        pixmaps       : 0
        pictures      : 0
        glyphsets     : 0
        colormaps     : 0
        passive grabs : 0
        cursors       : 0
        unknowns      : 0
        pixmap bytes  : 0
        other bytes   : ~48
        total bytes   : ~48
10 - <unknown> ( PID:  ?   ):
        res_base      : ox1400000
        res_mask      : ox1fffff
        windows       : 0
        GCs           : 1
        fonts         : 0
        pixmaps       : 0
        pictures      : 0
        glyphsets     : 0
        colormaps     : 0
        passive grabs : 0
        cursors       : 0
        unknowns      : 0
        pixmap bytes  : 0
        other bytes   : ~24
        total bytes   : ~24
11 - <unknown> ( PID:  ?   ):
        res_base      : ox200000
        res_mask      : ox1fffff
        windows       : 0
        GCs           : 1
        fonts         : 0
        pixmaps       : 0
        pictures      : 0
        glyphsets     : 0
        colormaps     : 0
        passive grabs : 0
        cursors       : 0
        unknowns      : 0
        pixmap bytes  : 0
        other bytes   : ~24
        total bytes   : ~2
  

As another followup, using my laptop with Xorg 1.6.1.901 and Intel 2.7.1 drivers, I get similar results: gvim will release most of its memory when I kill it, and Qt based applications will not release memory when they exit.

As an added bonus, Qt based applications will hang or crash the X server when too many are opened at once.  This occurs well below the normal 255 client limit, and the X server normally crashes out entirely.  X's memory usage does not have to be particularly huge for this to occur, even small windows like a kdialog can cause the crash while there is still free memory.

I went back in time with an Ubuntu 8.10 installation to see how it behaved, and got the same basic memory bug with the X.org 1.5 series and Qt 4.4.3 using the Intel driver without compositing.  The older software leaks, but overall RAM usage was not as great as with the newer versions which use more RAM and therefore leak a lot faster.

If valgrind --leak-check=full doesn't show where the apparent leaks are coming from, you could try running the X server in something like odin (http://cgit.freedesktop.org/~ickle/odin), which shows where memory was allocated at any time.

OK, 1 more drop into endless sea:
Hardware: GF6150 GO (mobile integrated chip)
2GB RAM.
no swapfile (why would i need it with 2 GB RAM?)
Software:
gentoo linux 2.6.31
nvidia-driver 185.31 - 195.25, nv driver from kernel (does not really matter)
X.Org X Server 1.6.3.901 (1.6.4 RC 1)
Release Date: 2009-8-25
X Protocol Version 11, Revision 0
Build Operating System: Linux 2.6.30-gentoo-r6-localbuild x86_64
Current Operating System: Linux headhunter.homelinux.org 2.6.31-gentoo-localbuild #2 SMP PREEMPT Sat Sep 19 19:44:00 EEST 2009 x86_64
Build Date: 17 September 2009  02:29:14AM
Of course, it is somehow beta-based system, but anyway, X seems to gradually consume RAM and CPU resources. After night of doing nothing X consumes about 200 MB, with active usage (opening and closing windows mostly affects) RAM usage gradually grows up to 1.6 GB and maybe more. CPU usage on memory allocation grows as well, so basically at 1.6 GB usage kernel has big troubles allocating big chunks of RAM (I have tried some 3D apps known to allocate big chunks (spring RTS) and they refused to start (ram allocation failed)). So basically X consumes and fragments hell amount of RAM without reason. It started with recent versions of X, but I can't tell which exactly since I don't usually leave my computer up for long. I'll do valgrind debug ASAP.

Ancient bug, closing.