Bug 87103 - Compilation is *very* slow.
Summary: Compilation is *very* slow.
Status: RESOLVED FIXED
Alias: None
Product: Mesa
Classification: Unclassified
Component: Drivers/DRI/i965 (show other bugs)
Version: 10.1
Hardware: x86-64 (AMD64) Linux (All)
: medium normal
Assignee: Intel 3D Bugs Mailing List
QA Contact: Intel 3D Bugs Mailing List
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: i965-perf
  Show dependency treegraph
 
Reported: 2014-12-08 14:55 UTC by Gonz
Modified: 2019-05-28 09:44 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gonz 2014-12-08 14:55:18 UTC
Version: Mesa 10.1.3 Ubuntu package
Chipset: HM87
System Architecture: Ubuntu x86 64 bit.
xf86-video-intel: 2:2.99.910-ubuntu1.3
xserver: 1.15.1
mesa: 10.1.3
libdrm: 2.4.52
kernel: 3.8.11
Distribution: Ubuntu
Machine: Acer C720 with Celeron 2955U.
Display connector: Intel Haswell Integrated graphics controller


The following shader takes minutes to compile:

const int n = 49;
void prod(in vec4 a[n], in vec4 b[n], out vec4 result[4]) {
                result[0] = vec4(0);
                for(int i = 0; i < n; ++i) {
                        result[0] += a[i] * b[i];
                }


It works with n = 26, but from n = 27 it is very slow.

I can't upgrade because intel-linux-graphics-installer bails out with

No LSB modules are available.
polkit: Fetching org.01.linuxgraphics.installer permissions...
diagnostics-view.c/diagnostics_view_start: Running diagnostic Checking if Intel graphics card available...
main-window.c/on_diagnostics_finished: Diagnostics finished with an error

Probably because I'm running a chroot Ubuntu on a Chromebook.
Comment 1 Gonz 2014-12-15 12:22:06 UTC
60% of the time is spent in:
brw::fs_live_variables::compute_live_variables()
Comment 2 Matt Turner 2014-12-15 17:45:33 UTC
Interesting.

I cannot reproduce the slow down on commit 52901ec from August, so perhaps this isn't a problem with more recent versions than 10.1.3.

I do, however, see some improvements we could make to the generated code.
Comment 3 Matt Turner 2014-12-15 18:51:43 UTC
(In reply to Matt Turner from comment #2)
> I do, however, see some improvements we could make to the generated code.

Oh, even that problem has been fixed since the commit I tested.

Please test a newer version. If a newer version isn't available in your distro, you can build Mesa and then set LD_LIBRARY_PATH and LIBGL_DRIVERS_PATH to the lib/ directory in the Mesa build tree to run your application against it.
Comment 4 Gonz 2014-12-15 23:20:55 UTC
I also tested on latest git; the problem persists.

From callgrind_annotate:

324,681,210,134  ../src/mesa/mesa/src/mesa/drivers/dri/i965/brw_fs_live_variables.cpp:brw::fs_live_variables::compute_live_variables() [../src/mesa/mesa/lib/i965_dri.so]
 30,407,730,617  ../src/mesa/mesa/src/mesa/drivers/dri/i965/brw_fs_live_variables.cpp:brw::fs_live_variables::compute_start_end() [../src/mesa/mesa/lib/i965_dri.so]
 27,294,134,491  ../src/mesa/mesa/src/util/register_allocate.c:ra_add_node_adjacency [../src/mesa/mesa/lib/i965_dri.so]
 16,474,809,991  ../src/mesa/mesa/src/mesa/drivers/dri/i965/brw_fs_reg_allocate.cpp:fs_visitor::assign_regs(bool) [../src/mesa/mesa/lib/i965_dri.so]

The compilation itself takes minutes, not the execution of the resulting shader.
Comment 5 Gonz 2014-12-15 23:25:28 UTC
52901ec
android: add CleanSpec.mk

Seems unrelated to compilaton times.
Comment 6 Matt Turner 2014-12-16 04:36:03 UTC
(In reply to Gonz from comment #5)
> 52901ec
> android: add CleanSpec.mk
> 
> Seems unrelated to compilaton times.

Yes... I wasn't claiming the commit fixed something. I was just saying that it was what I had installed.

Perhaps you can make a shader_test (see http://cgit.freedesktop.org/piglit/tree/tests/shaders/ssa/fs-if-def-else-break.shader_test?id=f18b947d7c03b81b989ed47551dc24bcc29cc0fc for example) that exhibits the problem?

I made one, but since it doesn't reproduce the problem I can't be sure of anything.
Comment 7 Gonz 2014-12-16 15:22:38 UTC
This one takes 13 seconds to compile. Change n to 70 or 80 and it gets worse.

[require]
GLSL >= 1.30

[fragment shader]
out vec4 fragColor;

const int n = 60;

vec4 a[n];
vec4 b[n];
vec4 c[4];

void init() {
    for(int i = 0; i < n; ++i) {
        a[i] = vec4(0.1, 0, 0, 0);
        b[i] = vec4(0.1, 0, 0, 0);
    }
}

void prod(in vec4 a[n], in vec4 b[n], out vec4 result[4]) {
	result[0] = vec4(0);
    for(int i = 0; i < n; ++i) {
        result[0] += a[i] * b[i];
    }
}

void main()
{
	init();
	prod(a, b, c);
    fragColor = c[0];
}

[test]

draw rect -1 -1 2 2
probe all rgba 0.6 0.0 0.0 0.0
Comment 8 Matt Turner 2014-12-16 16:59:19 UTC
Yes, that shader (n = 60) cannot be register allocated (at least with our current scheduling and allocation code) and ends up generating nearly 3600 instructions.
Comment 9 Gonz 2014-12-16 17:59:51 UTC
I would understand slow execution of that shader. But compilation?
So arrays with 60 elements are a WONTFIX?
Comment 10 Matt Turner 2014-12-16 18:43:52 UTC
It's not a WONTFIX, but I think it'll take some work.

The shader has enough live variables that it causes spilling. Each time we spill, we recalculate liveness. We could probably fix up the existing liveness information and save a bunch of time. We'd still generate awful code, though much more quickly.

The frontend compiler, based on some questionable heuristics and the lack of indirect addressing support in the i965 backend, lowers the indirect array accesses to a series of if statements, to do a binary search. We'd do significantly better to support indirect addressing and skip this stuff entirely.

It's on our TODO list, we just haven't gotten there yet.
Comment 11 Gonz 2014-12-17 10:54:26 UTC
Thanks for clarifying.
Comment 12 Eero Tamminen 2016-08-31 12:05:14 UTC
(In reply to Gonz from comment #7)
> This one takes 13 seconds to compile. Change n to 70 or 80 and it gets worse.

Gonz, on my SKL GT2, your shader takes 3 seconds to compile with latest Mesa.  Do you see similar improvement?
Comment 13 Eero Tamminen 2018-03-02 08:26:31 UTC
(In reply to Eero Tamminen from comment #12)
> (In reply to Gonz from comment #7)
> > This one takes 13 seconds to compile. Change n to 70 or 80 and it gets worse.
> 
> Gonz, on my SKL GT2, your shader takes 3 seconds to compile with latest
> Mesa.  Do you see similar improvement?

And now shader cache is now enabled, so it should take that time only on first startup.
Comment 14 Tapani Pälli 2019-05-28 09:44:25 UTC
Over the years, various changes on compiler have resulted in better compilation times. The shader test in comment #7 takes some seconds now with HSW, resolving this as fixed.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.