Bug 75871

Summary: systemd-networkd: could not bring up interface: Connection timed out (randomly)
Product: systemd Reporter: Gerardo Exequiel Pozzi <vmlinuz386>
Component: generalAssignee: Tom Gundersen <teg>
Status: RESOLVED FIXED QA Contact: systemd-bugs
Severity: normal    
Priority: medium CC: teg
Version: unspecified   
Hardware: x86-64 (AMD64)   
OS: Linux (All)   
Whiteboard:
i915 platform: i915 features:

Description Gerardo Exequiel Pozzi 2014-03-07 03:38:57 UTC
Sometimes randomly (~50% of times) networkd fails with this error message.
Under this state, only the address is configured, the gateway is missing. Restarting the service sets correctly the network.

--
Mar 07 00:29:44 exequiel systemd-udevd[169]: renamed network interface eth0 to enp0s16
Mar 07 00:29:57 exequiel kernel: forcedeth 0000:00:10.0 enp0s16: MSI enabled
**
Mar 07 00:29:58 exequiel systemd-networkd[328]: enp0s16: carrier on
Mar 07 00:29:58 exequiel systemd-networkd[328]: enp0s16: could not bring up interface: Connection timed out
Mar 07 00:29:58 exequiel systemd-networkd[328]: enp0s16: failed
--

# cat /etc/systemd/network/enp0s16.network 
[Match]
Name=enp0s16

[Network]
Description=red
Address=192.168.0.77/24
Gateway=192.168.0.1
#


Running 210-3 on Arch Linux (x86_64) custom kernel (3.10.25).
Network driver is forcedeth (built-in).
Comment 1 Tom Gundersen 2014-03-07 13:44:23 UTC
Could you try again with full debugging output. Drop debug.conf into /etc/systemd/system/systemd-networkd.service.d/ with the lines:

    [Service]
    Environment=SYSTEMD_LOG_LEVEL=debug

What I'm wondering about is the time stamps of the lines "enp0s16: bringing link up" and "enp0s16: could not bring up interface: Connection timed out" in the failing case, and in the successful case, the line "enp0s16: link is up".
Comment 2 Gerardo Exequiel Pozzi 2014-03-07 16:50:29 UTC
When enabling debug level, network is always configured right all times. I tried rebooting some times but always works 100%.

Mar 07 13:42:54 exequiel systemd-udevd[168]: renamed network interface eth0 to enp0s16
**
Mar 07 13:43:06 exequiel systemd-networkd[327]: timestamp of '/etc/systemd/network' changed
Mar 07 13:43:06 exequiel systemd-networkd[327]: timestamp of '/usr/lib/systemd/network' changed
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: found matching network '/etc/systemd/network/enp0s16.network'
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: requesting link status
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: bringing link up
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: setting addresses
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link (with ifindex 2) added
Mar 07 13:43:06 exequiel systemd-networkd[327]: lo: link (with ifindex 1) added
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: got link state
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link status updated: 00000000 -> 0x00001002
Mar 07 13:43:06 exequiel systemd-networkd[327]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus object=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 reply_c
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link is up
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: carrier on
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link status updated: 0x00001002 -> 0x00011003
Mar 07 13:43:06 exequiel systemd-networkd[327]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.2 object=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 error=n/a
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link status unchanged: 0x00011003
Mar 07 13:43:06 exequiel systemd-networkd[327]: Got message type=signal sender=org.freedesktop.DBus destination=:1.2 object=/org/freedesktop/DBus interface=org.freedesktop.DBus member=NameAcquired cookie=2 reply
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: addresses set
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: setting routes
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link status updated: 0x00011003 -> 0x00011043
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: routes set
Mar 07 13:43:06 exequiel systemd-networkd[327]: enp0s16: link configured
**
Mar 07 13:43:07 exequiel kernel: forcedeth 0000:00:10.0 enp0s16: MSI enabled
Comment 3 Tom Gundersen 2014-03-07 16:53:33 UTC
Hm, that's a shame. Could you at least include the full output of

    # journalctl -b -u systemd-networkd

in a failing case (even if debug is not enabled)?
Comment 4 Gerardo Exequiel Pozzi 2014-03-07 22:49:57 UTC
Yes, looks like "debug" add needed "delays", otherwise set to default "info", always fails. (Now I can not get at least one good network setup on boot)

# journalctl -b -u systemd-networkd
-- Logs begin at Sat 2013-12-14 14:49:00 ART, end at Fri 2014-03-07 19:45:09 ART. --
Mar 07 19:43:38 exequiel systemd-networkd[328]: enp0s16: carrier on
Mar 07 19:43:38 exequiel systemd-networkd[328]: enp0s16: could not bring up interface: Connection timed out
Mar 07 19:43:38 exequiel systemd-networkd[328]: enp0s16: failed
# 

Nothing new, just look like initial comment.
Comment 5 Tom Gundersen 2014-03-08 00:13:45 UTC
So as there is not more debug output available, I'm not sure if you are actually hitting the 10 second timeout, or if there is some other bug...

I just pushed two changes: 1) increased the default timeout to 25 secs (to match sd-bus); and 2) don't actually enter failed state just because the call to UP appears to have failed (there will still be a warning though).

Could you test with git if this fixes the problem for you, and if you still see a warning here?
Comment 6 Gerardo Exequiel Pozzi 2014-03-08 02:11:47 UTC
Building with 59a7a684 and 76800848 works fine now.

In all boot cases I get:
# journalctl -b -u systemd-networkd
-- Logs begin at Sat 2013-12-14 14:49:00 ART, end at Fri 2014-03-07 23:10:14 ART. --
Mar 07 23:07:13 exequiel systemd-networkd[326]: enp0s16: carrier on
Mar 07 23:07:13 exequiel systemd-networkd[326]: enp0s16: link configured
#

No warning appears, so looks like increasing timeout (first patch) in my case is sufficient, right?

Thanks you :)
Comment 7 Gerardo Exequiel Pozzi 2014-03-08 18:14:09 UTC
Now I also added a third patch 58b12917, recently commited by zbyszek and still works fine without any warning ;)

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.