Bug 29535

OK, I spent a while tonight to reproduce this, and then was surprised to find out that systemd actually does the right thing. ;-)

There are two reasons this doesn't work as expected:

- The initial transaction is calculated when systemd starts-up. systemd then compiles a list of things to do during bootup, looking at all dependencies. Those are then called "pending jobs" and kept around and bit by bit processed. When you disable a unit while we are processing the pending jobs (which does not make it completely unavailable, just causes it not to be considered when a transaction is compiled), then this has no influence on the jobs already pending. There's a very simple way to make sure that systemd also drops the queued jobs: just call "systemctl stop foobar.service" in addition to "systemctl disable foobar.service" and not only will the unit not be considered for future transactions anymore, but also a new transaction be schedule which contradicts the existing jobs and which hence causes them to be removed from the list of pending jobs. 

- The two .service files did not carry a service type field (i.e. Type= setting in [Service]). That means Type=simple was implied. In that case using ordering dependencies on it does not help for synchronization: systemd will just fork off the script and immediately continue with the next one. To do what you want to do you must make sure that systemd actually waits until all processes spawned for the service finished: use Type=oneshot for that (that's the systemd v8 name for it, in systemd v7 it was called Type=finish). Unfortunately the user must configure the type of the service manually, since we cannot autodect this. i.e. there is stuff like getty which just gets forked off, there's stuff like fsck, where we need to wait for completion and there's even stuff like traditional sysv daemons where we need to wait until the process we started exited and something was forked off in the background.

by making these two changes (i.e. placing "systemctl stop test-last.service" in /bin/test-first.sh and Type=oneshot in both .service files) things started to work as intended.

Thanks a lot for putting together this test case. Much appreciated!

I wonder if we can make this more discoverable, but tbh I am actually kinda convinced that the current behaviour of systemd here is pretty OK. What do you think?

Ah, yeah, that explains a lot. I kind of thought the problem was something like the initial transaction list you describe, but didn't think to try a 'stop' command as a way to deal with it. I also thought the service type may be important but couldn't quite figure out what to set it to either :)

Frankly I'm fine with this as long as there *is* a way to do it and it works. It would probably help to note it somewhere, maybe in that tips page johann's working on.

the only remaining issue is if we can do this with legacy services - in the case i'm mostly interested in, live boot, most of the service we want to disable are (currently) legacy sysv services. So the advice here doesn't map exactly. Would we do 'chkconfig foobar off' and then 'service foobar stop'? 'chkconfig foobar off' and then 'systemctl foobar.service stop'?

(In reply to comment #2)

> the only remaining issue is if we can do this with legacy services - in the
> case i'm mostly interested in, live boot, most of the service we want to
> disable are (currently) legacy sysv services. So the advice here doesn't map
> exactly. Would we do 'chkconfig foobar off' and then 'service foobar stop'?
> 'chkconfig foobar off' and then 'systemctl foobar.service stop'?

My recommendation is to always disable stuff first and then shut it down. It fixes a tiny race that something might get activated again in the time between it was stopped and disabled.

So my recommendation is something like this:

<snip>
systemctl disable foo.service
chkconfig foobar off
systemctl stop foo.service
service foobar stop
</snip>

In this order. And of course all of this directed to 2>&1 > /dev/null.

for the record, for anyone else who finds this bug in future, there's a key tweak needed to lennart's suggestion:

systemctl --no-reload disable foo.service
chkconfig foobar off
systemctl stop foo.service
service foobar stop

without --no-reload, the disable will trigger a reload of systemd, which can really screw boot up.