apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Programmatic log4j appender in Apex
Date Mon, 10 Apr 2017 21:30:23 GMT
As we are already doing dynamic log level changes and are talking about
dynamically adding appenders, does it make sense and technically feasible
to apply a delta log4j configuration dynamically (which can include log
levels + appeneders + ?) on top of the static configuration provided by the
system.

On Mon, Apr 10, 2017 at 2:18 PM, Sergey Golovko <sergey@datatorrent.com>
wrote:

> Ram,
>
> Really the new appender class must extend the abstract class
> AppenderSkeleton. And in order to add a new appender programmatically in
> Java, some code in Apex should call the following log4j method:
>
> org.apache.log4j.Logger.getRootLogger().addAppender(Appender newAppender)
>
> The general idea of my proposal is "*based on some runtime parameter(s) to
> provide ability to create an appender instance via reflection and add it to
> the list of active log4j appenders*".
>
> Thanks,
> Sergey
>
>
> On Mon, Apr 10, 2017 at 2:04 PM, Vlad Rozov <v.rozov@datatorrent.com>
> wrote:
>
> > It will require application recompilation and repackaging. The proposed
> > functionality is for dev-ops to be able to route application logging to a
> > preferred destination without recompiling applications. It is run-time
> > configuration vs compile time hardcoded appender.
> >
> > Thank you,
> >
> > Vlad
> >
> > On 4/10/17 11:23, Munagala Ramanath wrote:
> >
> >> You can do it in a trivial derived class without changing the base
> class.
> >>
> >> Ram
> >>
> >> On Mon, Apr 10, 2017 at 11:19 AM, Vlad Rozov <v.rozov@datatorrent.com>
> >> wrote:
> >>
> >> Does not the proposal to use Logger.addAppender() requires modifications
> >>> to used operators code?
> >>>
> >>> Thank you,
> >>>
> >>> Vlad
> >>>
> >>> On 4/10/17 10:58, Munagala Ramanath wrote:
> >>>
> >>> People can currently do this by simply implementing the Appender
> >>>> interface
> >>>> and adding it
> >>>> with Logger.addAppender() in the setup method. Why do we need
> something
> >>>> more elaborate ?
> >>>>
> >>>> Ram
> >>>>
> >>>> On Mon, Apr 10, 2017 at 10:30 AM, Sergey Golovko <
> >>>> sergey@datatorrent.com>
> >>>> wrote:
> >>>>
> >>>> The configuration of a log4j appender via log4j configuration file is
> a
> >>>>
> >>>>> static configuration that cannot be disabled/enabled and managed
> >>>>> dynamically by an application designer. The programmatic approach
> will
> >>>>> allow  an application designer to specify which of the available
> log4j
> >>>>> appenders should be used for the specific application.
> >>>>>
> >>>>> It is not necessary Apex should use the predefined log4j appenders
> >>>>> only.
> >>>>> The log4j events contain useful but the very limited number of
> >>>>> properties
> >>>>> which values can be printed into output log4j sources. But based
on
> the
> >>>>> knowledge of the software product workflow, the custom defined log4j
> >>>>> appender can extend a list of predefined output log events properties
> >>>>> and,
> >>>>> for instance for Apex, return: node, user name, application name,
> >>>>> application id, container id, operator name, etc.
> >>>>>
> >>>>> Also the output log events that are generated by a custom defined
> log4j
> >>>>> appender can be stored and indexed by any type of a full text search
> >>>>> database. It will allow the customers and developers to simplify
> >>>>> collection
> >>>>> of log events statistics and searching/filtering of specific events
> for
> >>>>> debugging and investigation.
> >>>>>
> >>>>> Thanks,
> >>>>> Sergey
> >>>>>
> >>>>>
> >>>>> On Mon, Apr 10, 2017 at 6:34 AM, Vlad Rozov <v.rozov@datatorrent.com
> >
> >>>>> wrote:
> >>>>>
> >>>>> +1 Apex engine does not own log4j config file - it is provided either
> >>>>> by
> >>>>>
> >>>>>> Hadoop or an application. Hadoop log4j config does not necessarily
> >>>>>> meet
> >>>>>> application logging requirements, but if log4j is provided by
an
> >>>>>> application designer, who can only specify what to log, it may
not
> >>>>>> meet
> >>>>>> operations requirements. Dev-ops should have an ability to specify
> >>>>>> where
> >>>>>>
> >>>>>> to
> >>>>>
> >>>>> log depending on the available infrastructure at run-time.
> >>>>>>
> >>>>>> It will be good to have an ability not only specify extra log4j
> >>>>>> appenders
> >>>>>> at lunch time, but also at run-time, the same way how log4j
logger
> >>>>>> levels
> >>>>>> may be changed.
> >>>>>>
> >>>>>> Thank you,
> >>>>>>
> >>>>>> Vlad
> >>>>>>
> >>>>>> On 4/9/17 23:14, Priyanka Gugale wrote:
> >>>>>>
> >>>>>> We can always write a custom appender and add it by changing
root
> >>>>>> appender
> >>>>>> in log4j config file. Can you explain how adding appender
> >>>>>> grammatically
> >>>>>>
> >>>>>>> would help?
> >>>>>>>
> >>>>>>> -Priyanka
> >>>>>>>
> >>>>>>> On Sun, Apr 9, 2017 at 11:50 AM, Sanjay Pujare <
> >>>>>>> sanjay@datatorrent.com
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Please give some examples and/or use cases of this programmatic
> log4j
> >>>>>>>
> >>>>>>> appender.
> >>>>>>>>
> >>>>>>>> On Fri, Apr 7, 2017 at 8:40 PM, Sergey Golovko <
> >>>>>>>> sergey@datatorrent.com
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> I'd like to add supporting of a custom defined log4j
appender that
> >>>>>>>>> can
> >>>>>>>>> be
> >>>>>>>>> added to Apex Application Master and Containers
and be
> configurable
> >>>>>>>>> programmatically.
> >>>>>>>>>
> >>>>>>>>> Sometimes it is not trivial to control log4j configuration
via
> >>>>>>>>> log4j
> >>>>>>>>> properties. And I think the having of the approach
to add a log4j
> >>>>>>>>>
> >>>>>>>>> appender
> >>>>>>>>>
> >>>>>>>> programmatically will allow the customers and developers
to plugin
> >>>>>>>> their
> >>>>>>>>
> >>>>>>> own custom defined log4j appenders and be much flexible
for
> streaming
> >>>>>>
> >>>>>>> and
> >>>>>>>>> collection of Apex log events.
> >>>>>>>>>
> >>>>>>>>> I assume to provide generic approach for definition
of the
> >>>>>>>>>
> >>>>>>>>> programmatic
> >>>>>>>>
> >>>>>>> log4j appender and to pass all configuration parameters
including a
> >>>>>>
> >>>>>>> name
> >>>>>>>>
> >>>>>>> of
> >>>>>>
> >>>>>>> the Java class with implementation of the log4j appender
via system
> >>>>>>>>
> >>>>>>>>> and/or
> >>>>>>>>>
> >>>>>>>> command line properties.
> >>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> Sergey
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message