apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AJAY GUPTA <ajaygit...@gmail.com>
Subject Re: [DISCUSS] Stram events log levels
Date Wed, 25 Jan 2017 07:15:38 GMT
Hi

For the initial release, I am planning to have static event log levels,
i.e., log levels without considering context in which event was generated.
I will add support for updating this log level so that in future, we can
update log levels considering the context of the event.


Ajay



On Fri, Jan 20, 2017 at 2:04 PM, AJAY GUPTA <ajaygit158@gmail.com> wrote:

> Nice point Tushar. I will look into how we can pass on different event log
> levels for same event type.
>
>
> Ajay
>
> On Fri, Jan 20, 2017 at 10:18 AM, Tushar Gosavi <tushar@datatorrent.com>
> wrote:
>
>> +1 for this feature. The event priority is not statically defained by
>> its type, it also depends on on context in which event is generated.
>> for example STOP_CONTAINER event could be because of
>>
>> - jvm issue/out of memory - This could be at much higher level
>> - All operators in containers have shutdown voluntary/or extra
>> container released - this could be at lower level.
>>
>> - Tushar.
>>
>>
>>
>> On Fri, Jan 20, 2017 at 12:32 AM, Sanjay Pujare <sanjay@datatorrent.com>
>> wrote:
>> > Strong +1 for the reasons cited by Ajay and Hitesh.
>> >
>> > Do we really need to modify the EventInfo class? Wouldn’t the
>> BeanUtils.describe(event) automatically populate the EventLogLevel into
>> “data” (a Map<String, String>) as a property? All the downstream code
>> including the REST API will then just return the EventLogLevel as part of
>> JSONObject without making any changes, right?
>> >
>> > Another thing: these levels should ideally match the corresponding
>> entries in the dt.log file as some users might expect.
>> >
>> > On 1/19/17, 10:10 AM, "Hitesh Kapoor" <hitesh@datatorrent.com> wrote:
>> >
>> >     +1 it will enhance the user experience and the approach suggested
>> by Ajay
>> >     doesn't seems to add significant overhead. Also I feel that in
>> future it
>> >     can be enhanced further and UI can also filter the event logs based
>> on
>> >     these log levels.
>> >
>> >     On Jan 19, 2017 10:57 PM, "AJAY GUPTA" <ajaygit158@gmail.com>
>> wrote:
>> >
>> >     > Current event log mechanism : The events recorded are currently
>> recorded in
>> >     > datatorrent/apps/{appid}/events/    folder in files named
>> part{id}.txt .
>> >     > eg
>> >     > : datatorrent/apps/application_1483801541352_0003/events/part0
>> .txt
>> >     >
>> >     > We can access these events via REST Api which returns the list of
>> events as
>> >     > List<EventInfo> objects.
>> >     >
>> >     >
>> >     > The below approach can be used for implementing this. Kindly let
>> me know
>> >     > your viewpoints/suggestions on the approach.
>> >     >
>> >     > *Required Changes:*
>> >     >
>> >     > a) Add a enum field like EventLogLevel in StramEvent, assign the
>> log level
>> >     > in constructors of different Stram events like StartOperator,
>> >     > StartContainer, StopOperator, etc.
>> >     >
>> >     > b) Add eventLogLevel in EventInfo class. The events returned via
>> REST api
>> >     > from StramClient is a list of EventInfo objects.
>> >     >
>> >     > c) Changes in EventsAgent - processPartFile() method to parse the
>> >     > eventLogLevel logged into the part{id}.txt file
>> >     >
>> >     >
>> >     > On Thu, Jan 19, 2017 at 10:55 PM, AJAY GUPTA <
>> ajaygit158@gmail.com> wrote:
>> >     >
>> >     > > Hi Apex community,
>> >     > >
>> >     > >
>> >     > > Currently, events logged by stram such as StartContainer,
>> StartOperator,
>> >     > > StopContainer, StopOperator, etc dont have any associated
>> priority level.
>> >     > >
>> >     > > We can provide log levels for such Stram events. Log levels can
>> be INFO,
>> >     > > WARN, ERROR
>> >     > > Eg:
>> >     > > 1. Start Container, Start Operator are INFO level events
>> >     > > 2. OperatorError is ERROR level event
>> >     > > 3. Stop Container, Stop Operator are WARN level events
>> >     > >
>> >     > > Log level for events can help in user experience when showing
>> the event
>> >     > > list in a GUI. eg: In datatorrent gateway UI, we can provide
>> color-coded
>> >     > > log levels so that user can focus more on ERROR and WARN events.
>> >     > >
>> >     > >
>> >     > >
>> >     > >
>> >     > >
>> >     >
>> >
>> >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message