apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Priyanka Gugale <priya...@datatorrent.com>
Subject Adding new log4j appender to Apex core
Date Tue, 22 Nov 2016 18:00:13 GMT

I am working on APEXCORE-563
As per this Jira we should put log file name in container/operator events.
The problem is current RollingFileAppender keeps renaming files from 1 to 2
to ... n as files reach maximum allowed file size. Because of constant
renaming of files we can't put a fixed file name in stram event.

To overcome this I would like to add a new log4j appender to ApexCore.
There are two ways I can implement this:
1. Have Daily rolling file appender. The current file will be recognized
based on timestamp in file name. Also to control max file size, we need to
keep rolling files based on size as well.
2. Have Rolling File Appender but do not rename files. When max file size
is reached create new file with next number e.g. crate log file dt.log.2
after dt.log.1 is full. Also to recognize the latest file keep the softlink
named dt.log pointing to current log file.

I would prefer to implement approach 2. Please provide your
comments/feedback if you feel otherwise.

Also to use this new appender we need to use our custom log4j.properties
file instead of one present in hadoop conf. For that we need to set jvm
option -Dlog4j.configuration. I am planning to update file dt-site.xml in
folder ~/.dt  and default properties file available in apex archetype to
set jvm options as follows:

And I will copy log4j.props file in ~/.dt folder as well as apex-archetypes.

Lastly if someone still miss this new log4j properties file or jvm option
to set -Dlog4j.configuration we will not put log file name in event raised
by container or operator.

Please provide your feedback on this approach.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message