Hello,

there seems to be a major memory leak in Xorg in combination with the VMware driver and the llvmpipe/softpipe renderer. It can be easily reproduced using glxgears. This works even on some real hardware systems when forcing software rendering (depending on the graphics driver which is used):

export GALLIUM_DRIVER="llvmpipe"
export LIBGL_ALWAYS_SOFTWARE="true"
glxgears -info

When glxgears is using llvmpipe or softpipe as renderer, it should be reproducible. Grab a corner of the glxgears window and quickly move it around (i.e., resize the window continuously) while watching Xorg memory which rapidly starts to grow. When stopping the resizing, memory usage stays constant and drops again after closing the window.

On my systems running in VirtualBox and vSphere, llvmpipe was used by default, so the exports are not necessary and the problem always occurs. In VirtualBox, the problem did not occur when I enabled 3D acceleration (VM settings -> Display -> Screen) as then SVGA3D is used as renderer.

I could observe this behavior on Ubuntu 18.04 (with stock Mesa 18.2.2 and also using the Ubuntu-X team PPA with Mesa 18.3.3), Ubuntu 18.10, Fedora 29 (Xwayland) and Debian 9, so I suspect it might be something in Mesa. I could not find an existing bug report that seems to match this problem so I opened this one.