Bug 90193

Summary: journalctl --directory doesn't follow rotated files properly
Product: systemd Reporter: Sebastian Schindler <sebastian.schindler+bugzilla>
Component: generalAssignee: systemd-bugs
Status: NEW --- QA Contact: systemd-bugs
Severity: normal    
Priority: medium    
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:
Attachments: short example to reproduce the error
file with switched entries
better example to reproduce the error
better example to reproduce the error
condensed version of the reproducer

Description Sebastian Schindler 2015-04-27 12:48:18 UTC
Created attachment 115371 [details]
short example to reproduce the error

If you follow a directory via

journalctl --directory /some/directory/ -f

and then a journal file is rotated, for some reason this file doesn't get followed properly.

Other, nonrotated files however are followed fine.

See the attached file for an example which should make the case more clear.

I reported something similar some weeks ago, but the example wasn't as good: https://bugs.freedesktop.org/show_bug.cgi?id=89764

89764 can be closed if this gets closed, I think.
Comment 1 Sebastian Schindler 2015-04-27 12:48:58 UTC
Created attachment 115372 [details]
file with switched entries
Comment 2 Sebastian Schindler 2015-04-27 12:49:50 UTC
Comment on attachment 115371 [details]
short example to reproduce the error

To create an example that triggers the rotation which leads to the bug I did the following steps:

-log something:
$logger a
$logger b

-save the coresponding journalentries:
$journalctl -n2 -o export >> entries.txt

-switch the timestamps and cursors of the entries (result named switched_entries.txt, see attachment)
-this will cause the journalfile to be rotated after the first entry "a" is written

-now follow a directory with journalctl
$journalctl --directory logtest/ -f

-create a new logfile with this entries:
$cat switched_entries.txt | systemd-journal-remote -o ~/logtest/a.journal -

-OUTPUT of journalctl:
No journal files were found.
Apr 27 10:07:40 virtual-fedora-sbs tpiadmin[9617]: a

-some new logging into the file:
$logger c
$journalctl -o export -n 1 | systemd-journal-remote -o ~/logtest/a.journal -

-leads to no new output, however:

-some logging into a new file in the same directory
$logger a2
$journalctl -o export -n 1 | systemd-journal-remote -o ~/logtest/b.journal -

-OUTPUT:
No journal files were found.
Apr 27 10:07:40 virtual-fedora-sbs tpiadmin[9617]: a
Apr 27 14:39:57 virtual-fedora-sbs tpiadmin[11424]: a2
//only the last line is actually new

-this new file will be followed until it gets rotated

-this problem also occurs, if the file gets rotated for other reasons than this "errors", for example if simply a rotation due to size happens



-some weird interaction I noticed: If the logs are put into the journal file two times:
$ journalctl -o export -n 1 | systemd-journal-remote -o ~/logtest/b.journal -
$ journalctl -o export -n 1 | systemd-journal-remote -o ~/logtest/b.journal -
-then the output of this doubled input is shown once, however the file isn't properly followed after that (new entries won't show up)
Comment 3 Sebastian Schindler 2015-04-27 12:51:15 UTC
Created attachment 115373 [details]
better example to reproduce the error
Comment 4 Sebastian Schindler 2015-04-27 12:52:12 UTC
Created attachment 115374 [details]
better example to reproduce the error
Comment 5 Lennart Poettering 2015-04-27 13:51:57 UTC
What backing file system is this?
Comment 6 Sebastian Schindler 2015-04-28 07:16:34 UTC
Everything described was tested on ext4.
Comment 7 Sebastian Schindler 2015-05-26 11:19:08 UTC
Created attachment 116042 [details]
condensed version of the reproducer

I rewrote the reproducer for clarity, now it should be easier to see the bug "in action". 
I should add that I used the (back then latest) systemd version 219.

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.