Bug 108946 - High memory usage in Black Mesa
Summary: High memory usage in Black Mesa
Alias: None
Product: Mesa
Classification: Unclassified
Component: Mesa core (show other bugs)
Version: 18.2
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: mesa-dev
QA Contact: mesa-dev
Depends on:
Reported: 2018-12-05 04:06 UTC by MGG
Modified: 2018-12-13 05:25 UTC (History)
2 users (show)

See Also:
i915 platform:
i915 features:

glxinfo (161.58 KB, text/plain)
2018-12-05 04:06 UTC, MGG
game apitrace (298.23 MB, application/octet-stream)
2018-12-05 04:09 UTC, MGG
Terminal Output upon executing bms.sh (242.88 KB, text/plain)
2018-12-11 21:46 UTC, Justin Mitzel
BMS apitrace with default arguments (78.17 MB, application/octet-stream)
2018-12-11 21:48 UTC, Justin Mitzel
BMS apitrace with -dev argument (96.60 MB, application/octet-stream)
2018-12-11 21:51 UTC, Justin Mitzel

Note You need to log in before you can comment on or make changes to this bug.
Description MGG 2018-12-05 04:06:12 UTC
Created attachment 142728 [details]


Since some time this game keeps getting an out of memory error that makes it crash (it's a 32bit binary). While reporting the issues to the game devs this is the reason they gave to the users (https://steamcommunity.com/app/362890/discussions/1/3307213006836757658/?ctp=7):

"... From what I noticed, more shaders game had - faster out of memory occurred. Black Mesa has full (or almost full) support of SM 3.0 on Linux which allowed us to use same shaders with same code on both platfroms, while opening new horizons for our new dynamic lighting implementation. After first discovery of issues with Mesa driver, to avoid out of memory I had to cut some of shader combinations completely specifically for OpenGL version by introducing shader platform flags that decide if particular part with some visual feature the of shader will be compiled. That helped quite a lot, we were able to run Black Mesa for a while on AMD with lowest settings and CSM being turned off. But while Mesa is sharing same memory location in application's address space, it's pretty hard to go further being 32 bit."

Also, they mention that with Nvidia propietary driver the game with full setting doesn't go over 1.2 GB of ram (on my system, I detect a first peak of almost 3Gb of ram usage before crashing). So, there is some way that Mesa drivers could be leaking about 2Gb of ram? I understand that the memory usage between propietary drivers a Mesa are quite different, but I feel that it's quite suspicious that there is such a big ram usage for what seems to be shaders compile output.

Finally, while running an apitrace I detected a lot this errors: "major shader compiler issue 21156: 0:4(12): warning: extension `GL_EXT_gpu_shader4' unsupported in vertex shader". Not sure if it could be related with any leak though.

I'm attaching to the ticket my system information plus and the apitrace I got until the crash. Let me know if there's any other information you need or if it's necessary that I test this game with newer version of Mesa.
Comment 1 MGG 2018-12-05 04:09:03 UTC
Created attachment 142729 [details]
game apitrace
Comment 2 Justin Mitzel 2018-12-11 21:44:53 UTC
I can also confirm that this is not a Mesa 18.2 specific bug as I am running Black Mesa on Mesa 19.0.0-devel. I will attach my own apitraces and terminal output and hopefully this can be solved soon as the BM dev says that this memory leak should be "Well known" to Mesa Devs.
Comment 3 Justin Mitzel 2018-12-11 21:46:12 UTC
Created attachment 142780 [details]
Terminal Output upon executing bms.sh
Comment 4 Justin Mitzel 2018-12-11 21:48:21 UTC
Created attachment 142781 [details]
BMS apitrace with default arguments
Comment 5 Justin Mitzel 2018-12-11 21:51:16 UTC
Created attachment 142782 [details]
BMS apitrace with -dev argument

-dev option creates static image on the menu-screen instead of an active scene without that argument
Comment 6 Ian Romanick 2018-12-12 04:55:52 UTC
(I'm quite sure I submitted a similar comment earlier today, but it seems to have just vanished.)

I ran the API trace in Valgrind massif memory profiler.  As far as I can tell, the application never calls glDeleteShader.  We don't release any of the memory because the app hasn't told us that we can.  In that trace there is is about 2.6GB (on a 64-bit build of apitrace) of intermediate IR that could be released by calling glDeleteShader on all the shaders that have been linked.

It's possible that this is a quirk of the API trace.  Someone would have to run the app with Valgrind massif to say for sure.

(In reply to MGG from comment #0)
> Finally, while running an apitrace I detected a lot this errors: "major
> shader compiler issue 21156: 0:4(12): warning: extension
> `GL_EXT_gpu_shader4' unsupported in vertex shader". Not sure if it could be
> related with any leak though.

This is unrelated.  It's a bit misleading that apitrace lists this as a "major shader compiler issue."  It is perfectly valid to enable extensions that the driver does not support.  GLSL is intentionally designed so that shaders can have multiple code paths that are used based on what extensions are available at compile time.
Comment 7 MGG 2018-12-13 05:25:54 UTC
Hello Ian,

I see your point about glDeleteShader, but due that the trace includes a game crash, I thinks it is quite possible that the api call is not present because the engine wasn't able to run any resource release process. Anyway, in my current setup I can make the game run for some time (at least in the main menu), so I'll run apitrace without letting the game to crash. I think that with a graceful run of the game we should be able to see if the leak is related with what you pointed out.

By the way, how useful would be to get an apitrace with an nvidia card (using the propietary driver)? Could that give you any extra information regarding the shader memory usage on each driver?

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.