Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Tue, 24 Mar 2015 01:13:54 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12757729.1416931376000.4483.1427159634451@Atlassian.JIRA>
In-Reply-To: <JIRA.12757729.1416931376000@Atlassian.JIRA>
References: <JIRA.12757729.1416931376000@Atlassian.JIRA>
 <JIRA.12757729.1416931376861@arcas>
Subject: [jira] [Commented] (YARN-2901) Add errors and warning stats to RM,
 NM web UI
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14377040#comment-14377040 ] 

Wangda Tan commented on YARN-2901:
----------------------------------

Hi [~vvasudev],

I spent some time take a look at Log4JMetricsAppeneder implementation (will include other modified component in next round).

1) Log4jMetricsAppender, 
1.1 Better to place in yarn-server-common?
1.2 If you agree above, how about put into package o.a.h.y.server.metrics (or utils)?
1.3 Rename it to Log4jWarnErrorMetricsAppender?
1.4 Comments about implementation:
I think currently, implementation of cleanup can be improved, now cutoff process of message/count is basically loop all items stored, which could be inefficient (imaging if number of stored message > threshold), existing logics in the patch would lead to lots of potential stored message (tons of messages could be genereated in 5 min, which is purge message task run interval).

If you can make the data structure to be:
SortedMap<String, SortedMap<Long, Integer>> errors (and warnings), the outside map is sorted by value (SortedMap with smallest timestamp goes first), and inside map is sorted by key (smallest timestamp goes first), purge can happen when we add any event, it will just take at most log(N=500) time to do the purge, and no extra timer task needed.

To make SortedMap can sort by value, one way to do that can refer to http://stackoverflow.com/questions/109383/how-to-sort-a-mapkey-value-on-the-values-in-java (first answer).

Here, value = SortedMap<Long, Integer>>, we can sort the SortedMaps according to smallest key in each SortedMap.

And one corner case may need to consider is, it is possible a same message can have lots of different timestamps, so we need purge the inner SortedMap too.

To make better code readability, you can wrap the SortedMap to a inner class like MessageInfo.

> Add errors and warning stats to RM, NM web UI
> ---------------------------------------------
>
>                 Key: YARN-2901
>                 URL: https://issues.apache.org/jira/browse/YARN-2901
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: Exception collapsed.png, Exception expanded.jpg, Screen Shot 2015-03-19 at 7.40.02 PM.png, apache-yarn-2901.0.patch, apache-yarn-2901.1.patch
>
>
> It would be really useful to have statistics on the number of errors and warnings in the RM and NM web UI. I'm thinking about -
> 1. The number of errors and warnings in the past 5 min/1 hour/12 hours/day
> 2. The top 'n'(20?) most common exceptions in the past 5 min/1 hour/12 hours/day
> By errors and warnings I'm referring to the log level.
> I suspect we can probably achieve this by writing a custom appender?(I'm open to suggestions on alternate mechanisms for implementing this).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)