Bug 80537 - sysv-generator incorrectly handles sysv services that require network-online: it makes network-online.target Want those services, causing them to be started unexpectedly
Summary: sysv-generator incorrectly handles sysv services that require network-online:...
Alias: None
Product: systemd
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: All All
: medium critical
Assignee: systemd-bugs
QA Contact: systemd-bugs
Depends on:
Reported: 2014-06-25 20:54 UTC by Adam Williamson
Modified: 2014-06-26 04:22 UTC (History)
2 users (show)

See Also:
i915 platform:
i915 features:


Description Adam Williamson 2014-06-25 20:54:09 UTC
This was discussed downstream in Fedora Rawhide: https://bugzilla.redhat.com/show_bug.cgi?id=1112908 (and previously on a mailing list thread). Myself and Bruno Wolff noted that remaining sysv services on our systems seemed to be being started, even though we had not explicitly enabled them and they did not appear as SXX(service) symlinks in /etc/rc*.d/ (which is what indicates them as 'enabled' so far as SysV is concerned).

I noticed that the affected services seem to appear in /run/systemd/generator.late/network-online.target.wants , and traced things out from there.

I found that if a file in /etc/init.d contains an LSB stanza like:

# Required-Start: $network

then, on boot, there will be a symlink to the sysv-generator converted service in /run/systemd/generator.late/network-online.target.wants . (This also applies to Should-Start and I think to NetworkManager-wait-online - I think this will be triggered by anything considered to match SPECIAL_NETWORK_ONLINE_TARGET in sysv-generator around line 515).

Poking through sysv-generator.c , I believe I figured out what's going on here. What it's actually trying to do is the converse: to implement the "service wants the network to be up" dependency expressed by "Required-Start: $network" or similar, create a symlink to network-online.target in foo.service.wants/ . But the argument ordering is wrong for at least *this* case.

generate_unit_file() does this, for s->wants relationships:

        STRV_FOREACH(p, s->wants) {
                r = add_symlink(*p, s->name);
                if (r < 0)
                        log_error_unit(s->name, "Failed to create 'Wants' symlink to %s: %s", *p, strerror(-r));

At least in the affected case, it's basically calling add_symlink(wanting service,wanted service) . but the logic of add_symlink() is add_symlink(from,to), so it winds up creating a symlink from wanting.service to wanted.service.wants/wanting.service - making wanted.service depend on wanting.service, exactly the opposite of what we want.

So, I wrote a patch to fix that, by simply inverting the order of the add_symlink() parameters:

                r = add_symlink(s->name, *p);

but I found a couple of problems. This gets the logic right for the affected case, but still doesn't actually work, because it winds up creating a symlink like this:

/run/systemd/generator.late/plague-server.service.wants/network-online.target -> /run/systemd/generator.late/network-online.target

it should point to /lib/systemd/system/network-online.target , but it's not smart enough to realize that. I also realized that sysv-generator uses the generate_unit_file() wants stuff in two other cases, and for at least one of those, the ordering is correct. e.g. I create a symlink:

/etc/rc3.d/S50foo -> /etc/init.d/foo

So setting the service foo to be 'enabled' for runlevel 3, in sysv language. Now when I boot, sysv-generator creates:

/run/systemd/generator.late/runlevel3.target.wants/foo.service -> /run/systemd/generator.late/foo.service

that's using the same bit of generate_unit_file(), but for *this* usage, the ordering is correct, and does what we actually want: foo will run in 'runlevel3'. So we can't simply invert the ordering of the add_symlink() call without causing other problems.

So I think we need to go to load_sysv() and look at the special handling of the network online target:

                                        if (streq(m, SPECIAL_NETWORK_ONLINE_TARGET) && !is_before) {
                                                /* the network-online target is special, as it needs to be actively pulled in */
                                                r = strv_extend(&s->after, m);
                                                if (r < 0)
                                                        return log_oom();
                                                r = strv_extend(&s->wants, m);
                                                if (r < 0)
                                                        return log_oom();

and do...something different there. But I'm afraid my C isn't quite up to handling that 'something different', I don't think. I hope I've correctly analyzed the problem at least, though.

Obviously this bug is fairly serious - it means all sysv services present on the system will be started any time network-online.target is "Wanted" by anything *else* (which is usually the case, on a typical system). This could even cause a security problem - imagine you have /etc/init.d/telnet ...
Comment 1 Adam Williamson 2014-06-25 21:07:08 UTC
Small correction to this part:

"This also applies to Should-Start and I think to NetworkManager-wait-online - I think this will be triggered by anything considered to match SPECIAL_NETWORK_ONLINE_TARGET in sysv-generator around line 515"

The bug really only affects the SysV "$network" service, I think, it does not affect NetworkManager-wait-online. I was fooled by /etc/init.d/network on Fedora, which has this stanza:

# Provides: $network
# Should-Start: iptables ip6tables NetworkManager-wait-online NetworkManager
# Short-Description: Bring up/down networking
# Description: Bring up/down networking

If the "Provides: $network" line is present, network.service will be in /run/systemd/generator.late/network-online.target.wants/ . If I remove that line, it is not.

I don't think that's correct either, but it's interestingly different. I believe that case is hitting the code around the comment:

                                                /* NB: SysV targets
                                                 * which are provided
                                                 * by a service are
                                                 * pulled in by the
                                                 * services, as an
                                                 * indication that the
                                                 * generic service is
                                                 * now available. This
                                                 * is strictly
                                                 * one-way. The
                                                 * targets do NOT pull
                                                 * in the SysV
                                                 * services! */

So...yeah, mo' problems.
Comment 2 Adam Williamson 2014-06-26 04:22:52 UTC
Thomas has pushed a fix to master:


I've tested it, using a Fedora Rawhide scratch build:


It seems to work well, fixes the problems I'd observed, and the patch makes sense to me. It should probably go to the 214-stable branch too.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.