Bug 77879 - RFE: journald to accept custom fields
Summary: RFE: journald to accept custom fields
Status: RESOLVED FIXED
Alias: None
Product: systemd
Classification: Unclassified
Component: general (show other bugs)
Version: unspecified
Hardware: Other All
: medium normal
Assignee: systemd-bugs
QA Contact: systemd-bugs
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-04-24 08:37 UTC by Duncan Innes
Modified: 2014-04-24 14:41 UTC (History)
0 users

See Also:
i915 platform:
i915 features:


Attachments

Description Duncan Innes 2014-04-24 08:37:47 UTC
Add the ability for journald to create custom fields & values in addition to simply storing the message text.

i.e. iptables current log output:

"MESSAGE" : "IN= OUT=eth0 SRC=A.B.C.D DST=W.X.Y.Z LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=23937 DF PROTO=UDP SPT=52894 DPT=53 LEN=44"

could be expanded to:

"MESSAGE" : "IN= OUT=eth0 SRC=A.B.C.D DST=W.X.Y.Z LEN=64 TOS=0x00 PREC=0x00 TTL=64 ID=23937 DF PROTO=UDP SPT=52894 DPT=53 LEN=44",
"IPTABLES_OUT" : "eth0",
"IPTABLES_SRC" : "A.B.C.D",
"IPTABLES_DST" : "W.X.Y.Z",
"IPTABLES_LEN" : "64",
"IPTABLES_TOS" : "0x00",
"IPTABLES_PREC" : "0x00",
"IPTABLES_TTL" : "64",
"IPTABLES_ID" : "23937",
"IPTABLES_PROTO" : "UDP",
"IPTABLES_SPT" : "52894",
"IPTABLES_DPT" : "53"

The storage of these custom fields would be entirely up to the logging source.  If iptables does not have the ability to write these extra fields, the standard log message would still be used.

The naming of the custom fields would be up to the various logging sources.

Reason:

Rich searching of log data without having to parse MESSAGE fields.  Especially when logs are pushed to a central server mechanism.  Some logged data (iptables, httpd, audit, etc) lends itself entirely to this extra granularity, but it could not be forced on all data.

In estates with hundreds or thousands of servers sending logs to a central store, the ability to quickly search all logs for "AUDIT_TYPE" : "authentication failure" and "AUDIT_USER" : "vulnerable" will show attempts to brute force a user password across the whole estate (I know there are better ways of detecting this specific example, but it's the best I can come up with right now).

While these fields can be parsed out of the message line at a later point, it would add value to the data to have the logging application separate out the data rather than relying on grok parsing (or other) at a later point, which may not capture all possibilities.

Thanks

Duncan
Comment 1 Lennart Poettering 2014-04-24 14:41:35 UTC
Hmm? Journal fields are supposed to be freely defined by applications, in whatever way they need it.

See the man page, first paragraph:

http://www.freedesktop.org/software/systemd/man/systemd.journal-fields.html


Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.