hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maysam Yabandeh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6982) nntop: top­-like tool for name node users
Date Fri, 05 Sep 2014 23:58:30 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123852#comment-14123852
] 

Maysam Yabandeh commented on HDFS-6982:
---------------------------------------

Thanks [~wheat9].

bq. What are the minimal changes in the hadoop side to enable this functionality?
The minimal change is a couple of lines to register TopMetrics with the hadoop metrics system.

{code}
    DefaultMetricsSystem.initialize("nntop");
    TopConfiguration conf = new TopConfiguration();
    TopMetrics.initSingleton(conf, "processName", "sessionId",
        TopUtil.getRequestedReportPeriods(conf));
{code}
Also a config change to register TopAuditLogger as the nn audit logger.

bq. Should rolling window reside in the NN?
The rolling window only provides light weight aggregation and this logic can also be in an
external process as it was suggested in the second architecture in the design doc. To transfer
the events from the nn to the rolling window residing in another process (or any other aggregation
service) the second architecture benefits from already existing audit logs. We also have been
using this approach at Twitter mostly to be reliable against the worst case scenarios and
have the recent top users retrievable even if the name node is not responsive. The down side
was the overhead of parsing the logs. Smaller clusters might also rather not having to maintain
an additional process to have access to the top users.


> nntop: top­-like tool for name node users
> -----------------------------------------
>
>                 Key: HDFS-6982
>                 URL: https://issues.apache.org/jira/browse/HDFS-6982
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Maysam Yabandeh
>            Assignee: Maysam Yabandeh
>         Attachments: HDFS-6982.patch, HDFS-6982.v2.patch, nntop-design-v1.pdf
>
>
> In this jira we motivate the need for nntop, a tool that, similarly to what top does
in Linux, gives the list of top users of the HDFS name node and gives insight about which
users are sending majority of each traffic type to the name node. This information turns out
to be the most critical when the name node is under pressure and the HDFS admin needs to know
which user is hammering the name node and with what kind of requests. Here we present the
design of nntop which has been in production at Twitter in the past 10 months. nntop proved
to have low cpu overhead (< 2% in a cluster of 4K nodes), low memory footprint (less than
a few MB), and quite efficient for the write path (only two hash lookup for updating a metric).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message