hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kihwal Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11742) Improve balancer usability after HDFS-8188
Date Fri, 05 May 2017 16:38:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15998536#comment-15998536
] 

Kihwal Lee commented on HDFS-11742:
-----------------------------------

Setting HDFS-11377 as a dependency. In most cases, the proposed change will make it unnecessary,
but it is still needed the case where the number of targets is bigger than max number of mover
threads. In this case, it won't be possible to serve all targets even by allocating only one
thread per target.

> Improve balancer usability after HDFS-8188
> ------------------------------------------
>
>                 Key: HDFS-11742
>                 URL: https://issues.apache.org/jira/browse/HDFS-11742
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>            Priority: Blocker
>         Attachments: balancer2.8.png, HDFS-11742.branch-2.8.patch, HDFS-11742.branch-2.patch,
HDFS-11742.trunk.patch
>
>
> We ran 2.8 balancer with HDFS-8818 on a 280-node and a 2,400-node cluster. In both cases,
it would hang forever after two iterations. The two iterations were also moving things at
a significantly lower rate. The hang itself is fixed by HDFS-11377, but the design limitation
remains, so the balancer throughput ends up actually lower.
> Instead of reverting HDFS-8188 as originally suggested, I am making a small change to
make it less error prone and more usable.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message