hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8824) Do not use small blocks for balancing the cluster
Date Fri, 14 Aug 2015 20:18:46 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697679#comment-14697679
] 

Hudson commented on HDFS-8824:
------------------------------

FAILURE: Integrated in Hadoop-trunk-Commit #8308 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8308/])
HDFS-8824. Do not use small blocks for balancing the cluster. (szetszwo: rev 2bc0a4f299fbd8035e29f62ce9cd22e209a62805)
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
* hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Balancer.java


> Do not use small blocks for balancing the cluster
> -------------------------------------------------
>
>                 Key: HDFS-8824
>                 URL: https://issues.apache.org/jira/browse/HDFS-8824
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>         Attachments: h8824_20150727b.patch, h8824_20150811b.patch
>
>
> Balancer gets datanode block lists from NN and then move the blocks in order to balance
the cluster.  It should not use the blocks with small size since moving the small blocks generates
a lot of overhead and the small blocks do not help balancing the cluster much.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message