apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohit Jotwani <mo...@datatorrent.com>
Subject Re: Adding new log4j appender to Apex core
Date Wed, 23 Nov 2016 08:01:55 GMT
+1 - Approach 2

Regards,
Mohit

On Wed, Nov 23, 2016 at 12:35 PM, AJAY GUPTA <ajaygit158@gmail.com> wrote:

> +1 for approach 2.
>
>
> Regards,
> Ajay
>
> On Wed, Nov 23, 2016 at 12:16 PM, David Yan <david@datatorrent.com> wrote:
>
> > The goal of this log4j appender is to provide a log offset and the fixed
> > name of the container log file (instead of apex.log becoming apex.log.1
> and
> > then apex.log.2, etc due to rotation) as part of an error STRAM event so
> > users can easily locate the log entries around the error.
> >
> > The user can override the appender, but in that case, the engine detects
> > that and will not include the log location as part of the STRAM event.
> >
> > David
> >
> > On Tue, Nov 22, 2016 at 7:10 PM, Priyanka Gugale <
> priyanka@datatorrent.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > Thomas,
> > > Yes log4j is ultimately owned by user, and they should be able to
> > override
> > > it. What I am trying to do is provide a default behavior for Apex. In
> > case
> > > user isn't using any logger of their own we should use this new
> appender
> > of
> > > Apex rather than using standard log4j appender as per hadoop config.
> > >
> > > Sanjay,
> > > Archetype is the good place to put this and I will add it there, but
> many
> > > time people won't use it. So I wanted to keep it at ~/.dt as well. Is
> > there
> > > any other default config folder for Apex?
> > >
> > > Also I am not relying on anything. If we fail to find config in app jar
> > or
> > > ~/.dt we are going to skip usage of this new appender.
> > >
> > > -Priyanka
> > >
> > > On Wed, Nov 23, 2016 at 5:58 AM, Sanjay Pujare <sanjay@datatorrent.com
> >
> > > wrote:
> > >
> > > > The only way to “enforce” this new appender is to update the
> archetypes
> > > > (apex-app-archetype and apex-conf-archetype under apex-core/ )  to
> use
> > > the
> > > > new ones as default. But there does not seem to be a way to enforce
> > this
> > > > for anyone not using the archetypes.
> > > >
> > > > I agree with not relying on ~/.dt in apex-core.
> > > >
> > > > On 11/22/16, 1:08 PM, "Thomas Weise" <thw@apache.org> wrote:
> > > >
> > > >     The log4j configuration is ultimately owned by the user, so how
> do
> > > you
> > > > want
> > > >     to enforce a custom appender?
> > > >
> > > >     I don't think that this should rely on anything in ~/.dt either
> > > >
> > > >     Thomas
> > > >
> > > >     On Tue, Nov 22, 2016 at 10:00 AM, Priyanka Gugale <
> > > > priyanka@datatorrent.com>
> > > >     wrote:
> > > >
> > > >     > Hi,
> > > >     >
> > > >     > I am working on APEXCORE-563
> > > >     > <https://issues.apache.org/jira/browse/APEXCORE-563>
> > > >     > As per this Jira we should put log file name in
> > container/operator
> > > > events.
> > > >     > The problem is current RollingFileAppender keeps renaming files
> > > from
> > > > 1 to 2
> > > >     > to ... n as files reach maximum allowed file size. Because of
> > > > constant
> > > >     > renaming of files we can't put a fixed file name in stram
> event.
> > > >     >
> > > >     > To overcome this I would like to add a new log4j appender to
> > > > ApexCore.
> > > >     > There are two ways I can implement this:
> > > >     > 1. Have Daily rolling file appender. The current file will be
> > > > recognized
> > > >     > based on timestamp in file name. Also to control max file size,
> > we
> > > > need to
> > > >     > keep rolling files based on size as well.
> > > >     > 2. Have Rolling File Appender but do not rename files. When max
> > > file
> > > > size
> > > >     > is reached create new file with next number e.g. crate log file
> > > > dt.log.2
> > > >     > after dt.log.1 is full. Also to recognize the latest file keep
> > the
> > > > softlink
> > > >     > named dt.log pointing to current log file.
> > > >     >
> > > >     > I would prefer to implement approach 2. Please provide your
> > > >     > comments/feedback if you feel otherwise.
> > > >     >
> > > >     > Also to use this new appender we need to use our custom
> > > > log4j.properties
> > > >     > file instead of one present in hadoop conf. For that we need
to
> > set
> > > > jvm
> > > >     > option -Dlog4j.configuration. I am planning to update file
> > > > dt-site.xml in
> > > >     > folder ~/.dt  and default properties file available in apex
> > > > archetype to
> > > >     > set jvm options as follows:
> > > >     >  <property>
> > > >     >    <name>dt.attr.CONTAINER_JVM_OPTIONS</name>
> > > >     >    <value>-Dlog4j.configuration=log4j.props</value>
> > > >     >  </property>
> > > >     >
> > > >     > And I will copy log4j.props file in ~/.dt folder as well as
> > > >     > apex-archetypes.
> > > >     >
> > > >     > Lastly if someone still miss this new log4j properties file or
> > jvm
> > > > option
> > > >     > to set -Dlog4j.configuration we will not put log file name in
> > event
> > > > raised
> > > >     > by container or operator.
> > > >     >
> > > >     > Please provide your feedback on this approach.
> > > >     >
> > > >     > -Priyanka
> > > >     >
> > > >
> > > >
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message