hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10534) NameNode WebUI should display DataNode usage rate with a certain percentile
Date Wed, 22 Jun 2016 23:26:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15345386#comment-15345386

Zhe Zhang commented on HDFS-10534:

Thanks [~lewuathe]! Latest patch LGTM overall. A few minors:
# Can we add a sanity check when loading the {{dfs.namenode.metrics.node-usage.percentile}}
config? Assert that it is between 0 and 1.
# Minor suggestion on the description of {{dfs.namenode.metrics.node-usage.percentile}} below.
This could be confusing so I think we should be a little more specific.
This setting specifies the percentile level to report node usage metrics. For example 0.95
means reporting the node usage for the 95th percentile of all DataNodes. If this setting is
at 0.95 and the reported node usage is 70%, it means 95% of DataNodes have a usage below 70%.
# To be consistent with the mathematical [definition | https://en.wikipedia.org/wiki/Percentile],
maybe we should also called it xth percentile on the html.
percentileInfo.put("name", StringUtils.format("%dth percentile",

I'm trying to start a NameNode with the patch to verify the new HTML. If you have one ready
it'd be great to add a screenshot to the JIRA.

Thanks for the nice work!

> NameNode WebUI should display DataNode usage rate with a certain percentile
> ---------------------------------------------------------------------------
>                 Key: HDFS-10534
>                 URL: https://issues.apache.org/jira/browse/HDFS-10534
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode, ui
>            Reporter: Zhe Zhang
>            Assignee: Kai Sasaki
>         Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, HDFS-10534.03.patch, HDFS-10534.04.patch
> In addition of *Min/Median/Max*, another meaningful metric for cluster balance is DN
usage rate at a certain percentile (e.g. 90 or 95). We should add a config option, and another
filed on NN WebUI, to display this.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message