Bug 90268 - (tmpfiles,journald) systemd 219 + linux 4.0.x = journal failing to append on boot
Summary: (tmpfiles,journald) systemd 219 + linux 4.0.x = journal failing to append on ...
Status: RESOLVED WORKSFORME
Alias: None
Product: systemd
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: x86-64 (AMD64) Linux (All)
: high critical
Assignee: systemd-bugs
QA Contact: systemd-bugs
URL: https://bugs.archlinux.org/task/44799
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-05-01 14:30 UTC by brent s.
Modified: 2017-10-27 18:04 UTC (History)
1 user (show)

See Also:
i915 platform:
i915 features:


Attachments

Description brent s. 2015-05-01 14:30:51 UTC
in some cases (i can't seem to narrow down which yet), systemd fails to create /run/log/journal. This then causes further failure of journald to fail to both:

-append to the volatile journal
-append to the persistent journal (if one exists)- no flushes, as the volatile journal is not appended

from what i understand, the volatile journal does not exist because it normally is written to /run/log/journal. on boot, the entries of /run/log/journal are copied over to the persistent journal in /var/log/journal/ (if persistence is enabled) (is this correct?).

this is occurring on fully updated/patched Arch Linux, x86_64.
this behaviour does not exist on kernel 3.19.3.

of three machines with systemd, i note the following:

machineA (desktop): does not create /run/log/journal; no new journal entries are created.

machineB (laptop): does not create /run/log/journal; HOWEVER, new journal entries SEEM to be appended/created.

machineC (firewall box): /run/log/journal IS created; new journal entries are added.

machineA and machineB share the exact same /etc/systemd/journald.conf (with no syslog):
[bts@workhorse systemd]$ egrep -Ev '^[[:space:]]*(#|$)' journald.conf 
[Journal]
Storage=persistent
Compress=yes
SystemMaxUse=100M
SystemMaxFileSize=20M


machineC, however, uses no persistence and uses a purely volatile journal (with syslog):
[root@comptroller systemd]# egrep -Ev '^[[:space:]]*(#|$)' journald.conf 
[Journal]
Storage=volatile
Seal=no
SystemMaxUse=50M
RuntimeMaxUse=50M
ForwardToSyslog=yes


as a temporary workaround, it seems that "systemctl restart systemd-journald" on the affected machine (machineA) seems to allow new journal entries. however, /run/log/journal is still not created during the restart.


let me know what logs/output/etc. you wish to see.


(cross-linking for reference: https://bugs.archlinux.org/task/44799 )
Comment 1 brent s. 2015-05-20 20:34:58 UTC
this issue continues to persist on linux 4.0.4.
Comment 2 brent s. 2015-05-20 20:36:07 UTC
increasing severity/importance due to this being a huge failure- i have no way of getting logging running reliably on this box and there has been no response.
Comment 3 brent s. 2015-05-28 05:47:37 UTC
as an update, this appears to occur under the following conditions:

-/var is mounted as a bindmount,
-on another disk from /

not sure which is the cause. my guess is the first, but i do this to avoid unnecessary writes to the SSD (which / resides on).

please fix.
Comment 4 raoul 2016-12-17 10:40:43 UTC
I have the same exact problem (with /var/log mounted as tmpfs to avoid unnecessary writes to the SSD disk). Apparently this issue has never been fixed.
Comment 5 Lennart Poettering 2017-10-27 18:04:27 UTC
The relevant code has been reworked substantially in more current systemd version. If this continues to be a problem with 235, please open a new issue on github, and let's continue the discussion there. Thank you.


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.