Bug 88996

Summary: systemd is ended by oom killer if MemoryLimit is exhausted.
Product: systemd Reporter: Alexander Haensch <alexander.haensch>
Component: generalAssignee: systemd-bugs
Status: RESOLVED NOTABUG QA Contact: systemd-bugs
Severity: normal    
Priority: medium    
Version: unspecified   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:
Attachments: patch to add some words in the manpage

Description Alexander Haensch 2015-02-05 13:44:11 UTC
I think this issue supports systemd-critics a lot and it should be fixed.
Adding the following lines to a systemd unit can cause a termination of the systemd service, which leads to unclean shutdown of the system.

[Service]
MemoryAccounting=True
MemoryLimit=1G

The reason is that the MemoryLimit is per default added to the system slice.
This should not be the default behavior as a simple badly written script can exhaust the given limit and the swap. The consequence is a not responsible systemd.

A) systemd should have its own slice, so that memory restictions can not bring systemd down.
B) per default, units should not add to the system slice.
Comment 1 Lennart Poettering 2015-02-10 19:43:15 UTC
Hmm? Not following?

If you set MemoryLimit=1G on a service unit, then this will be set on that unit's cgroup, not on any of the parent's cgroups, including those where systemd itself is located in.

I am not sure I grok what precisely you are trying to do say? Are you suggesting there is a bug that we apply the memory limit cgroup attribute incorrectly, to the wrong cgroup?
Comment 2 Alexander Haensch 2015-02-10 20:51:31 UTC
In my case the the program consumes all the RAM up to 1G and then starts to swap. 
In the moment all swap was consumed systemd was terminated.
I can be a but in the kernel.
I workaround by disabling all swap and creating a new slice for that service. In the moment everything is working fine.

Is there any chance to restart systemd if something like this happens?
Comment 3 Lennart Poettering 2015-02-10 20:55:03 UTC
The kernel OOM killer always excludes PID 1 one from its killing spree, for a good reason.

Maybe it's not the PID 1 systemd that was killed, but a user systemd instance?

How did you notice it got killed?
Comment 4 Alexander Haensch 2015-02-10 21:03:24 UTC
It can be, that just the user systemd instance was killed.
systemdctl, etc was not working anymore with some message that the connection timed out.

That would mean that it is possible to start the instance again, if the issue is happening again?
Comment 5 Lennart Poettering 2015-02-11 11:35:30 UTC
(In reply to Alexander Haensch from comment #4)
> It can be, that just the user systemd instance was killed.
> systemdctl, etc was not working anymore with some message that the
> connection timed out.

Well, that might have different reasons. For example, it might just be that dbus-daemon was killed, which might cause communication timeouts.

> That would mean that it is possible to start the instance again, if the
> issue is happening again?

Well, yeah, systemd --user (in form of the user@.service service) can be restarted any time.

Note that dbus-daemon cannot be restarted currently without losing all its connections.

Anyway, I don't think there's anything to fix here in systemd, hence closing.
Comment 6 Alexander Haensch 2015-02-11 13:08:55 UTC
Created attachment 113350 [details] [review]
patch to add some words in the manpage

While it maybe not a bug, i would recommend to add some words in the manpage to point out the possible implications on programs in the system.slice
Comment 7 Lennart Poettering 2015-02-11 16:39:00 UTC
I am sorry, but the change you attached is not right. 

We only set the cgroup attribute on the cgroups of the units you set it for. So, again: are you saying you set MemoryLimit for a unit and it got enforced on processes of a different unit?
Comment 8 Alexander Haensch 2015-02-11 17:13:08 UTC
Well indeed you are right, cgroups are applied correctly.
So i do not understand whats going on, system services should not be affected by cgroups of services..

Thanks for taking the time here.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.