Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Fri, 12 Sep 2014 02:42:35 +0000 (UTC)
From: "Maysam Yabandeh (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12738528.1409691195000.15560.1410489755195@Atlassian.JIRA>
In-Reply-To: <JIRA.12738528.1409691195000@Atlassian.JIRA>
References: <JIRA.12738528.1409691195000@Atlassian.JIRA>
 <JIRA.12738528.1409691195158@arcas>
Subject: =?utf-8?Q?[jira]_[Commented]_(HDFS-6982)_nntop?=
 =?utf-8?Q?:_top=C2=AD-like_tool_for_name_node_users?=
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/HDFS-6982?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D14131=
039#comment-14131039 ]=20

Maysam Yabandeh commented on HDFS-6982:
---------------------------------------

Thanks [~andrew.wang]. Yeah the main reason for starting nntop as a separat=
e process was to be able to freely ship the tool without tying it to the en=
tire hadoop release. We also wanted the tool to be available even in the wo=
rst case scenario that nn is not responding. However by plotting the report=
ed metrics, in my opinion that is not a concern any more since even when nn=
 is not responding we still have the traffic of the last minute(s) that lea=
d nn to such state.

> nntop: top=C2=AD-like tool for name node users
> -----------------------------------------
>
>                 Key: HDFS-6982
>                 URL: https://issues.apache.org/jira/browse/HDFS-6982
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Maysam Yabandeh
>            Assignee: Maysam Yabandeh
>         Attachments: HDFS-6982.patch, HDFS-6982.v2.patch, nntop-design-v1=
.pdf
>
>
> In this jira we motivate the need for nntop, a tool that, similarly to wh=
at top does in Linux, gives the list of top users of the HDFS name node and=
 gives insight about which users are sending majority of each traffic type =
to the name node. This information turns out to be the most critical when t=
he name node is under pressure and the HDFS admin needs to know which user =
is hammering the name node and with what kind of requests. Here we present =
the design of nntop which has been in production at Twitter in the past 10 =
months. nntop proved to have low cpu overhead (< 2% in a cluster of 4K node=
s), low memory footprint (less than a few MB), and quite efficient for the =
write path (only two hash lookup for updating a metric).


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)