Mailing-List: contact dev-help@apex.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@apex.apache.org
MIME-Version: 1.0
In-Reply-To: <CA+mX7fptL3R-gpbY4LwynFz6hxqO3wY6YdE6E03Btogun_4Wsg@mail.gmail.com>
References: <CAKt=akHtFVXXWJWjm8vfW0wctK442Cb=g40WXef9kuykVbCFVQ@mail.gmail.com>
 <CA+5xAo2CL=E1Xd83BVZQ3RCK8+rvL2Jg2Cum2mdWhTxEdKd0Fg@mail.gmail.com>
 <5A1733A7-5107-45FE-867B-FA815A586902@datatorrent.com> <CAKt=akGWoQA4iiFhZCmkOj6VCnPHpVPT3=gj+q_xLBxcGM+hOw@mail.gmail.com>
 <CAMqituNy-J6ZcTdJqPqy5_B-Xb2jGNbn=K-Jn2QMzdUqNW3uhg@mail.gmail.com> <CA+mX7fptL3R-gpbY4LwynFz6hxqO3wY6YdE6E03Btogun_4Wsg@mail.gmail.com>
From: Mohit Jotwani <mohit@datatorrent.com>
Date: Wed, 23 Nov 2016 13:31:55 +0530
Message-ID: <CAJrpbrWb2yvQ3n1+O6U-fHecHfM3K68o7uXJDVDGq45j5PUKkg@mail.gmail.com>
Subject: Re: Adding new log4j appender to Apex core
To: dev <dev@apex.apache.org>
Content-Type: multipart/alternative; boundary=001a1146378274f98a0541f34d70
archived-at: Wed, 23 Nov 2016 08:02:05 -0000

--001a1146378274f98a0541f34d70
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

+1 - Approach 2

Regards,
Mohit

On Wed, Nov 23, 2016 at 12:35 PM, AJAY GUPTA <ajaygit158@gmail.com> wrote:

> +1 for approach 2.
>
>
> Regards,
> Ajay
>
> On Wed, Nov 23, 2016 at 12:16 PM, David Yan <david@datatorrent.com> wrote=
:
>
> > The goal of this log4j appender is to provide a log offset and the fixe=
d
> > name of the container log file (instead of apex.log becoming apex.log.1
> and
> > then apex.log.2, etc due to rotation) as part of an error STRAM event s=
o
> > users can easily locate the log entries around the error.
> >
> > The user can override the appender, but in that case, the engine detect=
s
> > that and will not include the log location as part of the STRAM event.
> >
> > David
> >
> > On Tue, Nov 22, 2016 at 7:10 PM, Priyanka Gugale <
> priyanka@datatorrent.com
> > >
> > wrote:
> >
> > > Hi,
> > >
> > > Thomas,
> > > Yes log4j is ultimately owned by user, and they should be able to
> > override
> > > it. What I am trying to do is provide a default behavior for Apex. In
> > case
> > > user isn't using any logger of their own we should use this new
> appender
> > of
> > > Apex rather than using standard log4j appender as per hadoop config.
> > >
> > > Sanjay,
> > > Archetype is the good place to put this and I will add it there, but
> many
> > > time people won't use it. So I wanted to keep it at ~/.dt as well. Is
> > there
> > > any other default config folder for Apex?
> > >
> > > Also I am not relying on anything. If we fail to find config in app j=
ar
> > or
> > > ~/.dt we are going to skip usage of this new appender.
> > >
> > > -Priyanka
> > >
> > > On Wed, Nov 23, 2016 at 5:58 AM, Sanjay Pujare <sanjay@datatorrent.co=
m
> >
> > > wrote:
> > >
> > > > The only way to =E2=80=9Cenforce=E2=80=9D this new appender is to u=
pdate the
> archetypes
> > > > (apex-app-archetype and apex-conf-archetype under apex-core/ )  to
> use
> > > the
> > > > new ones as default. But there does not seem to be a way to enforce
> > this
> > > > for anyone not using the archetypes.
> > > >
> > > > I agree with not relying on ~/.dt in apex-core.
> > > >
> > > > On 11/22/16, 1:08 PM, "Thomas Weise" <thw@apache.org> wrote:
> > > >
> > > >     The log4j configuration is ultimately owned by the user, so how
> do
> > > you
> > > > want
> > > >     to enforce a custom appender?
> > > >
> > > >     I don't think that this should rely on anything in ~/.dt either
> > > >
> > > >     Thomas
> > > >
> > > >     On Tue, Nov 22, 2016 at 10:00 AM, Priyanka Gugale <
> > > > priyanka@datatorrent.com>
> > > >     wrote:
> > > >
> > > >     > Hi,
> > > >     >
> > > >     > I am working on APEXCORE-563
> > > >     > <https://issues.apache.org/jira/browse/APEXCORE-563>
> > > >     > As per this Jira we should put log file name in
> > container/operator
> > > > events.
> > > >     > The problem is current RollingFileAppender keeps renaming fil=
es
> > > from
> > > > 1 to 2
> > > >     > to ... n as files reach maximum allowed file size. Because of
> > > > constant
> > > >     > renaming of files we can't put a fixed file name in stram
> event.
> > > >     >
> > > >     > To overcome this I would like to add a new log4j appender to
> > > > ApexCore.
> > > >     > There are two ways I can implement this:
> > > >     > 1. Have Daily rolling file appender. The current file will be
> > > > recognized
> > > >     > based on timestamp in file name. Also to control max file siz=
e,
> > we
> > > > need to
> > > >     > keep rolling files based on size as well.
> > > >     > 2. Have Rolling File Appender but do not rename files. When m=
ax
> > > file
> > > > size
> > > >     > is reached create new file with next number e.g. crate log fi=
le
> > > > dt.log.2
> > > >     > after dt.log.1 is full. Also to recognize the latest file kee=
p
> > the
> > > > softlink
> > > >     > named dt.log pointing to current log file.
> > > >     >
> > > >     > I would prefer to implement approach 2. Please provide your
> > > >     > comments/feedback if you feel otherwise.
> > > >     >
> > > >     > Also to use this new appender we need to use our custom
> > > > log4j.properties
> > > >     > file instead of one present in hadoop conf. For that we need =
to
> > set
> > > > jvm
> > > >     > option -Dlog4j.configuration. I am planning to update file
> > > > dt-site.xml in
> > > >     > folder ~/.dt  and default properties file available in apex
> > > > archetype to
> > > >     > set jvm options as follows:
> > > >     >  <property>
> > > >     >    <name>dt.attr.CONTAINER_JVM_OPTIONS</name>
> > > >     >    <value>-Dlog4j.configuration=3Dlog4j.props</value>
> > > >     >  </property>
> > > >     >
> > > >     > And I will copy log4j.props file in ~/.dt folder as well as
> > > >     > apex-archetypes.
> > > >     >
> > > >     > Lastly if someone still miss this new log4j properties file o=
r
> > jvm
> > > > option
> > > >     > to set -Dlog4j.configuration we will not put log file name in
> > event
> > > > raised
> > > >     > by container or operator.
> > > >     >
> > > >     > Please provide your feedback on this approach.
> > > >     >
> > > >     > -Priyanka
> > > >     >
> > > >
> > > >
> > > >
> > > >
> > >
> >
>

--001a1146378274f98a0541f34d70--