Bug 86815 - XWayland [weston 1.6.0] takes all CPU and is very slow on eee-pc
Summary: XWayland [weston 1.6.0] takes all CPU and is very slow on eee-pc
Status: RESOLVED DUPLICATE of bug 88316
Alias: None
Product: Wayland
Classification: Unclassified
Component: XWayland (show other bugs)
Version: unspecified
Hardware: Other All
: medium normal
Assignee: Wayland bug list
QA Contact: Xorg Project Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-11-28 08:22 UTC by Mildred Ki'Lya
Modified: 2015-12-17 09:12 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments
Configure glamor in weston.ini (2.23 KB, patch)
2015-01-07 13:52 UTC, Mildred Ki'Lya
Details | Splinter Review

Description Mildred Ki'Lya 2014-11-28 08:22:40 UTC
Running weston with the xwayland.so module on current ArchLinux (weston 1.6.0-1) cause on some computers XWayland to take all the CPU available (100% of one CPU core out of the two cores available on the system), and is very slow. X11 windows are shown but unusable.

This happens on a eee-pc 1005PE. Processor is dual core intel atom N450 with integrated intel graphics. On another computer running the same distro (and same package versions) but with an i7 processor, I have no problems, and XWayland CPU usage is more than correct.

I noticed the following pattern:

- running an X11 application cause XWayland to consume the most CPU
- after some time focusing on a wayland native application, XWayland stops eating CPU
- if focusing on the X11 window again (or simply hovering the mouse), CPU consumption rise again to 100%
- wayland surfaces continue to be responsive (that's nice actually)
- I notice glitches on X11 window decorations (the title bar disappear, then reappear). That may be unrelated though.

Looking at the Xwayland process using strace, I found a unusual number of SIGALRM signals being catched by Xwayland. I have the following sequence repeated many many many times:

--- SIGALRM {si_signo=SIGALRM, si_code=SI_KERNEL} ---
rt_sigreturn() = (number that varies a lot, 0, 1 and higher integers)

There are some ioctl in the mix and very few other system calls
Comment 1 Pekka Paalanen 2014-11-28 08:42:13 UTC
Hi, I'm not sure I can help you, but using 'perf' to try to see where exactly that CPU is used when hogging it would be nice.
https://perf.wiki.kernel.org/index.php/Tutorial#Processor-wide_mode_2

That would be 'perf record -a', do the CPU hog, hit ctrl+c for perf to stop it, and then you have a perf.data file to inspect with 'perf report'.

If you want to attach the perf.data file to this bug, then please also run 'perf archive' and attach the resulting perf.data.tar.bz2 or such too.
Comment 2 Mildred Ki'Lya 2014-12-08 12:51:22 UTC
Most time seems to be spent in i915_dri.so. There are many many symbols listed there, and none of them is above 5%.

The perf report is 25MB, so I won't include it. It doesn't have symbols for i915_dri (I'll install them and regenerate the report). The test was:

- run xterm in weston-terminal
- wait for xterm to show up, CPU is 100%
- click on weston-terminal to get focus on it, don't mouse over xterm any more
- wait for CPU usage to settle to something acceptable.
Comment 3 Mildred Ki'Lya 2014-12-08 13:01:58 UTC
Hardware as reported by lspci is:

00:02.0 VGA compatible controller: Intel Corporation Atom Processor D4xx/D5xx/N4xx/N5xx Integrated Graphics Controller
00:02.1 Display controller: Intel Corporation Atom Processor D4xx/D5xx/N4xx/N5xx Integrated Graphics Controller

Device id is respectively 8086:a011 and 8086:a012
Comment 4 Mildred Ki'Lya 2014-12-08 15:55:50 UTC
perf data gathered over the few seconds of running xterm inside weston is available here:

http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2

The most time-consuming routines are:

Overhead Command  Shared Object     Symbol
28.77%   Xwayland i915_dri.so       [.] fetch_vector4
19.88%   swapper  [kernel.kallsyms] [k] intel_idle
15.43%   Xwayland i915_dri.so       [.] _mesa_execute_program
12.57%   Xwayland i915_dri.so       [.] store_vector4

Other routines are below 5%

I suspect some buffers are copied too many times around in the memory. Would that be a problem in the intel driver of in Xwayland itself?
Comment 5 Mildred Ki'Lya 2014-12-08 16:03:26 UTC
Could this be related somehow to the version of OpenGL supported by this intel driver? This card only supports OpenGL 1.4.
Comment 6 Pekka Paalanen 2014-12-09 08:53:16 UTC
(In reply to Mildred Ki'Lya from comment #4)
> perf data gathered over the few seconds of running xterm inside weston is
> available here:
> 
> http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2

Unfortunately this package contains only the program binaries needed study the trace deeper, but not the trace itself. Yeah, 'perf archive' does that for some reason.

(In reply to Mildred Ki'Lya from comment #5)
> Could this be related somehow to the version of OpenGL supported by this
> intel driver? This card only supports OpenGL 1.4.

Really? I'm surprised glamor runs at all then. Yes, I suspect that is the issue, the glamor fallbacks could be very heavy on that OpenGL version and an Atom CPU, that you would probably be better off without glamor.

The next thing to try would be to disable glamor in Xwayland. I think the following patch to Weston should do it:

diff --git a/xwayland/launcher.c b/xwayland/launcher.c
index df2efd2..5471db0 100644
--- a/xwayland/launcher.c
+++ b/xwayland/launcher.c
@@ -116,6 +116,7 @@ weston_xserver_handle_event(int listen_fd, uint32_t mask, void *data)
                          "-listen", unix_fd,
                          "-wm", wm_fd,
                          "-terminate",
+                         "-shm",
                          NULL) < 0)
                        weston_log("exec of '%s %s -rootless "
                                    "-listen %s -listen %s -wm %s "

That is, just add "-shm" to the argument list. I'm not aware of any easier way to do it.

Let us know how that affects. If it works for you, then we have two possible items to fix:
- make forcing glamor off easier in Weston
- patch Xwayland server or glamor to revert to "shm" on such low GL support
Comment 7 Mildred Ki'Lya 2014-12-11 09:05:57 UTC
Well, removing the -shm option didnt help. I did it by creating a bash script in place of Xwayland to exec to the real binary, quicker to do than recompiling weston. I checked using a process manager the option was effectively gone.

I double checked my story about OpenGL 1.4 and I was wrong. I misinterpreted it with the glx version string within glxinfo. I get on native Xorg:

client glx version string: 1.4 (that awfully looks like a version of OpenGL to me)
server glx version string: 1.4
GLX version: 1.4
OpenGL vendor string: Intel Open Source Technology Center
OpenGL renderer string: Mesa DRI Intel(R) IGD
OpenGL version string: 2.1 Mesa 10.3.5
OpenGL shading language version string: 1.20

glxinfo on Xwayland will show:

OpenGL vendor string: VMWare, Inc.
OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.5, 128 bits)
OpenGL version string: 3.0 Mesa 10.3.5
OpenGL shading language version string: 1.30

I made available the perf.data file as:

http://mildredkilya.nfshost.com/perf-20141208.data

perf archive result is still available at:

http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2

tell me if you need any other file.
Comment 8 Pekka Paalanen 2014-12-11 13:21:17 UTC
(In reply to Mildred Ki'Lya from comment #7)
> Well, removing the -shm option didnt help.

Um, I did write "add", not "remove". :-)

Adding -shm would disable glamor.

> I double checked my story about OpenGL 1.4 and I was wrong. I misinterpreted
> it with the glx version string within glxinfo. I get on native Xorg:
> 
> OpenGL vendor string: Intel Open Source Technology Center
> OpenGL renderer string: Mesa DRI Intel(R) IGD
> OpenGL version string: 2.1 Mesa 10.3.5
> OpenGL shading language version string: 1.20

Ok, so there is at least GLSL support in theory. Much better.

> glxinfo on Xwayland will show:
> 
> OpenGL vendor string: VMWare, Inc.
> OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.5, 128 bits)

This just means that GL apps under Xwayland would be software rendered, but I'm not sure it implies anything about Glamor.

> OpenGL version string: 3.0 Mesa 10.3.5
> OpenGL shading language version string: 1.30
> 
> I made available the perf.data file as:
> 
> http://mildredkilya.nfshost.com/perf-20141208.data
> 
> perf archive result is still available at:
> 
> http://mildredkilya.nfshost.com/perf-20141208.data.tar.bz2
> 
> tell me if you need any other file.

Looks like I can't make sense of the trace. Maybe I can't use 'perf report' right?

All I see is a huge number of lines like
   1.47%         Xwayland  ^E                  [.] 0x0000000000211da8
and some rare others.
Comment 9 Mildred Ki'Lya 2014-12-23 15:44:10 UTC
Sorry, I haven't been back on this for some time. I added the "-shm" option instead of removing it, and now xterms is as fast as it could be. That works.

> Looks like I can't make sense of the trace. Maybe I can't use 'perf report' right?

When looking at the trace itself from my side, I gets tons of functions in Xwayland. See comment #4 for a partial transcript of what I see.

And you're seeing hexadecimal encoded address of symbols instead of the symbols themselves. I don't know how perf could package the symbol names so you could see them.

The top symbols that are slowing things down are fetch_vector4, intel_idle, _mesa_execute_program and store_vector4 (the only above 5%). I'm guessing that there is non necessary memory copying that happens. Storing and fetching vectors uselessly, and waiting idely for the process to complete.

If you know how to use perf report, you could tell me what you are looking for and I could get you the output I have.
Comment 10 Pekka Paalanen 2015-01-02 11:05:04 UTC
(In reply to Mildred Ki'Lya from comment #9)
> Sorry, I haven't been back on this for some time. I added the "-shm" option
> instead of removing it, and now xterms is as fast as it could be. That works.

Okay, so the problem is triggered by Glamor, I believe.

> The top symbols that are slowing things down are fetch_vector4, intel_idle,
> _mesa_execute_program and store_vector4 (the only above 5%). I'm guessing
> that there is non necessary memory copying that happens. Storing and
> fetching vectors uselessly, and waiting idely for the process to complete.

Unfortunately that goes out of my personal expertise.

Maybe the Intel GPU just does not have the hardware to do some thing and therefore they are done on the CPU?
Maybe the GL driver (in Mesa) is doing something unoptimal?
Or maybe Glamor is using GL in ways that cause the above?
Or could Glamor simply be a bad fit for this hardware and there should be some automatic way to disable Glamor at runtime?

I really don't know.

If you need more weight to make a case, so that Intel/Glamor developers would look into it, you could try running X.org with the modesetting DDX and Glamor. That is, nothing Wayland nor Weston. If that has similar slowness, you have a stronger case and could file it against Glamor or Intel. Whatever solution comes from there should also apply to Xwayland.
Comment 11 Zhigang Gong 2015-01-07 01:59:28 UTC
(In reply to Pekka Paalanen from comment #10)
> (In reply to Mildred Ki'Lya from comment #9)
> > Sorry, I haven't been back on this for some time. I added the "-shm" option
> > instead of removing it, and now xterms is as fast as it could be. That works.
> 
> Okay, so the problem is triggered by Glamor, I believe.
> 
> > The top symbols that are slowing things down are fetch_vector4, intel_idle,
> > _mesa_execute_program and store_vector4 (the only above 5%). I'm guessing
> > that there is non necessary memory copying that happens. Storing and
> > fetching vectors uselessly, and waiting idely for the process to complete.
> 
> Unfortunately that goes out of my personal expertise.
> 
> Maybe the Intel GPU just does not have the hardware to do some thing and
> therefore they are done on the CPU?

You may be right here. This Eee PC is using pineview platform which
doesn't have hardware vertex shader support. So all the vertex shader
fallbacks to CPU side, it becomes extremly slow then.

> Maybe the GL driver (in Mesa) is doing something unoptimal?
> Or maybe Glamor is using GL in ways that cause the above?
> Or could Glamor simply be a bad fit for this hardware and there should be
> some automatic way to disable Glamor at runtime?
> 
> I really don't know.
> 
> If you need more weight to make a case, so that Intel/Glamor developers
> would look into it, you could try running X.org with the modesetting DDX and
> Glamor. That is, nothing Wayland nor Weston. If that has similar slowness,
> you have a stronger case and could file it against Glamor or Intel. Whatever
> solution comes from there should also apply to Xwayland.
Comment 12 Mildred Ki'Lya 2015-01-07 13:52:35 UTC
Created attachment 111912 [details] [review]
Configure glamor in weston.ini

I posted this patch to the mailing list as well. it should add an option in weston.ini to make it possible to easily disable glamor.
Comment 13 Pekka Paalanen 2015-01-11 08:05:39 UTC
I do wonder if there should also be a bug report against Glamor saying that Glamor should automatically disable itself on this kind of hardware. After all, this kind of settings should be right without user configuration too.

But yeah, I think the idea of a weston.ini option is useful anyway.
Comment 14 Pekka Paalanen 2015-02-02 13:17:55 UTC
The related Glamor bug #88316.

And, there is a patch proposed:
http://lists.x.org/archives/xorg-devel/2015-January/045456.html
Comment 15 mercuriete 2015-02-20 20:37:26 UTC
I have exactly the same problem.
i am trying to recompile xorg-server without glamor support.

anyway looking the ebuild from portage (i am using gentoo) the xorg-server glamor support is not using lib glamor but mesa with egl and libepoxy instead.

http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/x11-base/xorg-server/xorg-server-1.16.4.ebuild?view=markup


i have an old acer aspire 5510 with the intel i915 graphic card.

i think there's a problem with the driver.


PD: while i was writing this comment finally i get the xorg-server without glamor support and this works.

now weston + xwayland runs smothly.
Comment 16 Olivier Fourdan 2015-12-17 09:12:42 UTC
I think this is a duplicate of bug 88316

Please reopen if the issue still occurs with xwayland from xserver 1.18.x

*** This bug has been marked as a duplicate of bug 88316 ***


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.