Bug 86815

Hi, I'm not sure I can help you, but using 'perf' to try to see where exactly that CPU is used when hogging it would be nice.
https://perf.wiki.kernel.org/index.php/Tutorial#Processor-wide_mode_2

That would be 'perf record -a', do the CPU hog, hit ctrl+c for perf to stop it, and then you have a perf.data file to inspect with 'perf report'.

If you want to attach the perf.data file to this bug, then please also run 'perf archive' and attach the resulting perf.data.tar.bz2 or such too.

Most time seems to be spent in i915_dri.so. There are many many symbols listed there, and none of them is above 5%.

The perf report is 25MB, so I won't include it. It doesn't have symbols for i915_dri (I'll install them and regenerate the report). The test was:

- run xterm in weston-terminal
- wait for xterm to show up, CPU is 100%
- click on weston-terminal to get focus on it, don't mouse over xterm any more
- wait for CPU usage to settle to something acceptable.

Hardware as reported by lspci is:

00:02.0 VGA compatible controller: Intel Corporation Atom Processor D4xx/D5xx/N4xx/N5xx Integrated Graphics Controller
00:02.1 Display controller: Intel Corporation Atom Processor D4xx/D5xx/N4xx/N5xx Integrated Graphics Controller

Device id is respectively 8086:a011 and 8086:a012

perf data gathered over the few seconds of running xterm inside weston is available here:

http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2

The most time-consuming routines are:

Overhead Command  Shared Object     Symbol
28.77%   Xwayland i915_dri.so       [.] fetch_vector4
19.88%   swapper  [kernel.kallsyms] [k] intel_idle
15.43%   Xwayland i915_dri.so       [.] _mesa_execute_program
12.57%   Xwayland i915_dri.so       [.] store_vector4

Other routines are below 5%

I suspect some buffers are copied too many times around in the memory. Would that be a problem in the intel driver of in Xwayland itself?

Could this be related somehow to the version of OpenGL supported by this intel driver? This card only supports OpenGL 1.4.

(In reply to Mildred Ki'Lya from comment #4)
> perf data gathered over the few seconds of running xterm inside weston is
> available here:
> 
> http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2

Unfortunately this package contains only the program binaries needed study the trace deeper, but not the trace itself. Yeah, 'perf archive' does that for some reason.

(In reply to Mildred Ki'Lya from comment #5)
> Could this be related somehow to the version of OpenGL supported by this
> intel driver? This card only supports OpenGL 1.4.

Really? I'm surprised glamor runs at all then. Yes, I suspect that is the issue, the glamor fallbacks could be very heavy on that OpenGL version and an Atom CPU, that you would probably be better off without glamor.

The next thing to try would be to disable glamor in Xwayland. I think the following patch to Weston should do it:

diff --git a/xwayland/launcher.c b/xwayland/launcher.c
index df2efd2..5471db0 100644
--- a/xwayland/launcher.c
+++ b/xwayland/launcher.c
@@ -116,6 +116,7 @@ weston_xserver_handle_event(int listen_fd, uint32_t mask, void *data)
                          "-listen", unix_fd,
                          "-wm", wm_fd,
                          "-terminate",
+                         "-shm",
                          NULL) < 0)
                        weston_log("exec of '%s %s -rootless "
                                    "-listen %s -listen %s -wm %s "

That is, just add "-shm" to the argument list. I'm not aware of any easier way to do it.

Let us know how that affects. If it works for you, then we have two possible items to fix:
- make forcing glamor off easier in Weston
- patch Xwayland server or glamor to revert to "shm" on such low GL support

Well, removing the -shm option didnt help. I did it by creating a bash script in place of Xwayland to exec to the real binary, quicker to do than recompiling weston. I checked using a process manager the option was effectively gone.

I double checked my story about OpenGL 1.4 and I was wrong. I misinterpreted it with the glx version string within glxinfo. I get on native Xorg:

client glx version string: 1.4 (that awfully looks like a version of OpenGL to me)
server glx version string: 1.4
GLX version: 1.4
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) IGD
OpenGL version string: 2.1 Mesa 10.3.5
OpenGL shading language version string: 1.20

glxinfo on Xwayland will show:

OpenGL vendor string: VMWare, Inc.
OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.5, 128 bits)
OpenGL version string: 3.0 Mesa 10.3.5
OpenGL shading language version string: 1.30

I made available the perf.data file as:

http://mildredkilya.nfshost.com/perf-20141208.data

perf archive result is still available at:

http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2

tell me if you need any other file.

(In reply to Mildred Ki'Lya from comment #7)
> Well, removing the -shm option didnt help.

Um, I did write "add", not "remove". :-)

Adding -shm would disable glamor.

> I double checked my story about OpenGL 1.4 and I was wrong. I misinterpreted
> it with the glx version string within glxinfo. I get on native Xorg:
> 
> OpenGL vendor string: Intel Open Source Technology Center
> OpenGL renderer string: Mesa DRI Intel(R) IGD
> OpenGL version string: 2.1 Mesa 10.3.5
> OpenGL shading language version string: 1.20

Ok, so there is at least GLSL support in theory. Much better.

> glxinfo on Xwayland will show:
> 
> OpenGL vendor string: VMWare, Inc.
> OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.5, 128 bits)

This just means that GL apps under Xwayland would be software rendered, but I'm not sure it implies anything about Glamor.

> OpenGL version string: 3.0 Mesa 10.3.5
> OpenGL shading language version string: 1.30
> 
> I made available the perf.data file as:
> 
> http://mildredkilya.nfshost.com/perf-20141208.data
> 
> perf archive result is still available at:
> 
> http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2
> 
> tell me if you need any other file.

Looks like I can't make sense of the trace. Maybe I can't use 'perf report' right?

All I see is a huge number of lines like
   1.47%         Xwayland  ^E                  [.] 0x0000000000211da8
and some rare others.

Sorry, I haven't been back on this for some time. I added the "-shm" option instead of removing it, and now xterms is as fast as it could be. That works.

> Looks like I can't make sense of the trace. Maybe I can't use 'perf report' right?

When looking at the trace itself from my side, I gets tons of functions in Xwayland. See comment #4 for a partial transcript of what I see.

And you're seeing hexadecimal encoded address of symbols instead of the symbols themselves. I don't know how perf could package the symbol names so you could see them.

The top symbols that are slowing things down are fetch_vector4, intel_idle, _mesa_execute_program and store_vector4 (the only above 5%). I'm guessing that there is non necessary memory copying that happens. Storing and fetching vectors uselessly, and waiting idely for the process to complete.

If you know how to use perf report, you could tell me what you are looking for and I could get you the output I have.

(In reply to Mildred Ki'Lya from comment #9)
> Sorry, I haven't been back on this for some time. I added the "-shm" option
> instead of removing it, and now xterms is as fast as it could be. That works.

Okay, so the problem is triggered by Glamor, I believe.

> The top symbols that are slowing things down are fetch_vector4, intel_idle,
> _mesa_execute_program and store_vector4 (the only above 5%). I'm guessing
> that there is non necessary memory copying that happens. Storing and
> fetching vectors uselessly, and waiting idely for the process to complete.

Unfortunately that goes out of my personal expertise.

Maybe the Intel GPU just does not have the hardware to do some thing and therefore they are done on the CPU?
Maybe the GL driver (in Mesa) is doing something unoptimal?
Or maybe Glamor is using GL in ways that cause the above?
Or could Glamor simply be a bad fit for this hardware and there should be some automatic way to disable Glamor at runtime?

I really don't know.

If you need more weight to make a case, so that Intel/Glamor developers would look into it, you could try running X.org with the modesetting DDX and Glamor. That is, nothing Wayland nor Weston. If that has similar slowness, you have a stronger case and could file it against Glamor or Intel. Whatever solution comes from there should also apply to Xwayland.

(In reply to Pekka Paalanen from comment #10)
> (In reply to Mildred Ki'Lya from comment #9)
> > Sorry, I haven't been back on this for some time. I added the "-shm" option
> > instead of removing it, and now xterms is as fast as it could be. That works.
> 
> Okay, so the problem is triggered by Glamor, I believe.
> 
> > The top symbols that are slowing things down are fetch_vector4, intel_idle,
> > _mesa_execute_program and store_vector4 (the only above 5%). I'm guessing
> > that there is non necessary memory copying that happens. Storing and
> > fetching vectors uselessly, and waiting idely for the process to complete.
> 
> Unfortunately that goes out of my personal expertise.
> 
> Maybe the Intel GPU just does not have the hardware to do some thing and
> therefore they are done on the CPU?

You may be right here. This Eee PC is using pineview platform which
doesn't have hardware vertex shader support. So all the vertex shader
fallbacks to CPU side, it becomes extremly slow then.

> Maybe the GL driver (in Mesa) is doing something unoptimal?
> Or maybe Glamor is using GL in ways that cause the above?
> Or could Glamor simply be a bad fit for this hardware and there should be
> some automatic way to disable Glamor at runtime?
> 
> I really don't know.
> 
> If you need more weight to make a case, so that Intel/Glamor developers
> would look into it, you could try running X.org with the modesetting DDX and
> Glamor. That is, nothing Wayland nor Weston. If that has similar slowness,
> you have a stronger case and could file it against Glamor or Intel. Whatever
> solution comes from there should also apply to Xwayland.

Created attachment 111912 [details] [review]
Configure glamor in weston.ini

I posted this patch to the mailing list as well. it should add an option in weston.ini to make it possible to easily disable glamor.

I do wonder if there should also be a bug report against Glamor saying that Glamor should automatically disable itself on this kind of hardware. After all, this kind of settings should be right without user configuration too.

But yeah, I think the idea of a weston.ini option is useful anyway.

The related Glamor bug #88316.

And, there is a patch proposed:
http://lists.x.org/archives/xorg-devel/2015-January/045456.html

I have exactly the same problem.
i am trying to recompile xorg-server without glamor support.

anyway looking the ebuild from portage (i am using gentoo) the xorg-server glamor support is not using lib glamor but mesa with egl and libepoxy instead.

http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/x11-base/xorg-server/xorg-server-1.16.4.ebuild?view=markup


i have an old acer aspire 5510 with the intel i915 graphic card.

i think there's a problem with the driver.


PD: while i was writing this comment finally i get the xorg-server without glamor support and this works.

now weston + xwayland runs smothly.

I think this is a duplicate of bug 88316

Please reopen if the issue still occurs with xwayland from xserver 1.18.x

*** This bug has been marked as a duplicate of bug 88316 ***