hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10534) NameNode WebUI should display DataNode usage rate with a certain percentile
Date Fri, 24 Jun 2016 16:32:16 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15348540#comment-15348540

Andrew Wang commented on HDFS-10534:

Adding a new metric is compatible, but upon looking at this patch, I had a few questions.

* Why not present a histogram rather than a single threshold like this? That way we don't
add a new config, present more info, and don't require a restart to change this threshold.
* This is also a metric that could be calculated in client-side JS from existing information.
* Finally, the config says it's a percentile, but it's really a quantile. Percentile are 0-100,
quantiles are 0-1.0. Seems like the precondition check should also be inclusive of 0 and 1.0,
since they are valid quantiles.

I'd prefer if we backed out this change while we thought about these issues. Thanks folks.

> NameNode WebUI should display DataNode usage rate with a certain percentile
> ---------------------------------------------------------------------------
>                 Key: HDFS-10534
>                 URL: https://issues.apache.org/jira/browse/HDFS-10534
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode, ui
>            Reporter: Zhe Zhang
>            Assignee: Kai Sasaki
>             Fix For: 3.0.0-alpha1
>         Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, HDFS-10534.03.patch, HDFS-10534.04.patch,
HDFS-10534.05.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png
> In addition of *Min/Median/Max*, another meaningful metric for cluster balance is DN
usage rate at a certain percentile (e.g. 90 or 95). We should add a config option, and another
filed on NN WebUI, to display this.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message