apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Munagala Ramanath <...@datatorrent.com>
Subject Re: Programmatic log4j appender in Apex
Date Mon, 10 Apr 2017 23:18:09 GMT
http://logging.apache.org/log4j/1.2/faq.html#3.6

Log4j has the ability to dynamically reload changes to the properties file
via the
*configureAndWatch()* method of *PropertyConfigurator*. If this is already
baked in, could
devops simply change the properties file if we provide an enhanced set of
appenders
for the desired use cases ?

Ram

On Mon, Apr 10, 2017 at 2:04 PM, Vlad Rozov <v.rozov@datatorrent.com> wrote:

> It will require application recompilation and repackaging. The proposed
> functionality is for dev-ops to be able to route application logging to a
> preferred destination without recompiling applications. It is run-time
> configuration vs compile time hardcoded appender.
>
> Thank you,
>
> Vlad
> On 4/10/17 11:23, Munagala Ramanath wrote:
>
>> You can do it in a trivial derived class without changing the base class.
>>
>> Ram
>>
>> On Mon, Apr 10, 2017 at 11:19 AM, Vlad Rozov <v.rozov@datatorrent.com>
>> wrote:
>>
>> Does not the proposal to use Logger.addAppender() requires modifications
>>> to used operators code?
>>>
>>> Thank you,
>>>
>>> Vlad
>>>
>>> On 4/10/17 10:58, Munagala Ramanath wrote:
>>>
>>> People can currently do this by simply implementing the Appender
>>>> interface
>>>> and adding it
>>>> with Logger.addAppender() in the setup method. Why do we need something
>>>> more elaborate ?
>>>>
>>>> Ram
>>>>
>>>> On Mon, Apr 10, 2017 at 10:30 AM, Sergey Golovko <
>>>> sergey@datatorrent.com>
>>>> wrote:
>>>>
>>>> The configuration of a log4j appender via log4j configuration file is a
>>>>
>>>>> static configuration that cannot be disabled/enabled and managed
>>>>> dynamically by an application designer. The programmatic approach will
>>>>> allow  an application designer to specify which of the available log4j
>>>>> appenders should be used for the specific application.
>>>>>
>>>>> It is not necessary Apex should use the predefined log4j appenders
>>>>> only.
>>>>> The log4j events contain useful but the very limited number of
>>>>> properties
>>>>> which values can be printed into output log4j sources. But based on the
>>>>> knowledge of the software product workflow, the custom defined log4j
>>>>> appender can extend a list of predefined output log events properties
>>>>> and,
>>>>> for instance for Apex, return: node, user name, application name,
>>>>> application id, container id, operator name, etc.
>>>>>
>>>>> Also the output log events that are generated by a custom defined log4j
>>>>> appender can be stored and indexed by any type of a full text search
>>>>> database. It will allow the customers and developers to simplify
>>>>> collection
>>>>> of log events statistics and searching/filtering of specific events for
>>>>> debugging and investigation.
>>>>>
>>>>> Thanks,
>>>>> Sergey
>>>>>
>>>>>
>>>>> On Mon, Apr 10, 2017 at 6:34 AM, Vlad Rozov <v.rozov@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>> +1 Apex engine does not own log4j config file - it is provided either
>>>>> by
>>>>>
>>>>>> Hadoop or an application. Hadoop log4j config does not necessarily
>>>>>> meet
>>>>>> application logging requirements, but if log4j is provided by an
>>>>>> application designer, who can only specify what to log, it may not
>>>>>> meet
>>>>>> operations requirements. Dev-ops should have an ability to specify
>>>>>> where
>>>>>>
>>>>>> to
>>>>>
>>>>> log depending on the available infrastructure at run-time.
>>>>>>
>>>>>> It will be good to have an ability not only specify extra log4j
>>>>>> appenders
>>>>>> at lunch time, but also at run-time, the same way how log4j logger
>>>>>> levels
>>>>>> may be changed.
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Vlad
>>>>>>
>>>>>> On 4/9/17 23:14, Priyanka Gugale wrote:
>>>>>>
>>>>>> We can always write a custom appender and add it by changing root
>>>>>> appender
>>>>>> in log4j config file. Can you explain how adding appender
>>>>>> grammatically
>>>>>>
>>>>>>> would help?
>>>>>>>
>>>>>>> -Priyanka
>>>>>>>
>>>>>>> On Sun, Apr 9, 2017 at 11:50 AM, Sanjay Pujare <
>>>>>>> sanjay@datatorrent.com
>>>>>>> wrote:
>>>>>>>
>>>>>>> Please give some examples and/or use cases of this programmatic
log4j
>>>>>>>
>>>>>>> appender.
>>>>>>>>
>>>>>>>> On Fri, Apr 7, 2017 at 8:40 PM, Sergey Golovko <
>>>>>>>> sergey@datatorrent.com
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi All,
>>>>>>>>
>>>>>>>> I'd like to add supporting of a custom defined log4j appender
that
>>>>>>>>> can
>>>>>>>>> be
>>>>>>>>> added to Apex Application Master and Containers and be
configurable
>>>>>>>>> programmatically.
>>>>>>>>>
>>>>>>>>> Sometimes it is not trivial to control log4j configuration
via
>>>>>>>>> log4j
>>>>>>>>> properties. And I think the having of the approach to
add a log4j
>>>>>>>>>
>>>>>>>>> appender
>>>>>>>>>
>>>>>>>> programmatically will allow the customers and developers
to plugin
>>>>>>>> their
>>>>>>>>
>>>>>>> own custom defined log4j appenders and be much flexible for streaming
>>>>>>
>>>>>>> and
>>>>>>>>> collection of Apex log events.
>>>>>>>>>
>>>>>>>>> I assume to provide generic approach for definition of
the
>>>>>>>>>
>>>>>>>>> programmatic
>>>>>>>>
>>>>>>> log4j appender and to pass all configuration parameters including
a
>>>>>>
>>>>>>> name
>>>>>>>>
>>>>>>> of
>>>>>>
>>>>>>> the Java class with implementation of the log4j appender via
system
>>>>>>>>
>>>>>>>>> and/or
>>>>>>>>>
>>>>>>>> command line properties.
>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Sergey
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>
>>
>


-- 

_______________________________________________________

Munagala V. Ramanath

Software Engineer

E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam

www.datatorrent.com  |  apex.apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message