Bug 70661

Summary: add activity log to plugins
Product: LibreOffice Reporter: Norbert Thiebaud <nthiebaud>
Component: ci-infraAssignee: DavidO <d.ostrovsky>
Status: RESOLVED FIXED QA Contact:
Severity: enhancement    
Priority: medium    
Version: unspecified   
Hardware: Other   
OS: All   
Whiteboard:
i915 platform: i915 features:

Description Norbert Thiebaud 2013-10-20 05:17:38 UTC 
In order to help monitor what the tinderbox-client are doing
it is desirable to have an activity log for the buildbot plugin, and for the tb3-wrapper plugin.

This feature will be directed by an entry in the plugin's cfg that specify the location of the activity log file.
The presence of that entry trigger the generation of such logs
The log file does not need to exist, but the directory in which it is must exist
Data is appended to the logfiles.
The logfile should not be kept open for longer than necessary, and the logfile is susceptible to 'disappear' at any time (logrotate without sighup)
Logging should be made on a best effort basis.
On starting/loading of the plugin, a special 'start plugin' message is issued. if logging is not possible (file access error, missing directory etc..) a warning is issued in the gerrit error log, but the plugin still try to log after that, but does not warn anymore

On stop of the plugin an activity message is issued using the same format as the load message with s/start/stop/

The content of the logfile is text, one line per event with the folowing format

<iso-8601 datetime in UTC>:<ssh_user invoking>:<plugin_name>:<plugin command line>

'\n' character(s) in <command line>, if present, must be replaced by ' ' to keep the message in one text line.

<username> is assumed to be without ':'. If that assumption is not guaranteed, then ':' in the <username> must be substituted by '_'

example
2013-10-20T00:39Z:nthiebaud:buildbot:get master -p core --platform LINUX --id tb21

Attention: the datetime _must_ be in UTC, for every messages.

the 'start plugin' message is of the form
<iso-8601 datetime in UTC>::<plugin_name>:>start

the 'stop plugin' message is of the form
<iso-8601 datetime in UTC>::<plugin_name>:>stop


PS: yes <plugin_name> is required because multiple plugins may point to the same activity logfile.
Comment 1 DavidO 2013-10-20 11:02:55 UTC 
Done.

$>cat /home/gerrit2/gerrit2_lo/logs/shell_activity.log
2013-10-20 09:18:08,618::shell:>start
2013-10-20 09:18:14,807:davido:shell:/home/gerrit2/bin/say-hello --user davido --groups 'Administrators' Norbert -a foo bar
2013-10-20 09:18:43,282::shell:>stop

Documentation:

activity loggin is using standard Java log4j facility:
http://logging.apache.org/log4j/2.x/manual

Configuration is controlled by:
/home/gerrit2/gerrit2_lo/etc/log4j.properties

Custom logging configuration must be activated in Gerrit (only supported from Gerrit 2.8),
in [container] section in $site/gerrit.config -Dlog4j.configuration must be set:

[container]
        javaOptions = -Dlog4j.configuration=file:///home/gerrit2/gerrit2_lo/etc/log4j.properties

In log4j.properties section the appender and category must be activated:

log4j.logger.shell.activity_log=INFO, shell_appender
log4j.additivity.shell.activity_log=false

log4j.appender.shell_appender=org.apache.log4j.DailyRollingFileAppender
log4j.appender.shell_appender.datePattern='-'yyyy-MM'.log'
log4j.appender.shell_appender.File=logs/shell_activity.log
log4j.appender.shell_appender.layout=org.apache.log4j.PatternLayout
log4j.appender.shell_appender.layout.ConversionPattern=%d{ISO8601}:%m%n

See log4j manual for further infomration,
particularly appender and coonversation sections:

http://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/PatternLayout.html
http://logging.apache.org/log4j/2.x/manual/layouts.html
Comment 2 DavidO 2013-10-20 18:35:16 UTC 
Another IRC session:

* don't use rolling file appender: Done.
* don't have ':' in timestamp: Done.
* Ensure that the timestamp is UTC: Done

Currently the log4j configuration is as follow:

log4j.appender.logfile=org.apache.log4j.FileAppender
log4j.appender.logfile.File=logs/error.log
log4j.appender.logfile.layout=org.apache.log4j.EnhancedPatternLayout
log4j.appender.logfile.layout.ConversionPattern=%d{yyyyMMdd HHmmss,SSS}{UTC}:%m%n

and the result:

20131020 182942,415::shell:>start
20131020 183427,230:davido:shell:/home/gerrit2/bin/say-hello --user davido --groups 'Administrators' -a foo bar

Use of freedesktop.org services, including Bugzilla, is subject to our Code of Conduct. How we collect and use information is described in our Privacy Policy.