hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8824) Do not use small blocks for balancing the cluster
Date Fri, 07 Oct 2016 18:03:20 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhe Zhang updated HDFS-8824:
----------------------------
    Fix Version/s: 2.7.4

Thanks Nicholas for the work. I think this is a good improvement for branch-2.7 and just backported.
Verified {{TestBalancer}}.

> Do not use small blocks for balancing the cluster
> -------------------------------------------------
>
>                 Key: HDFS-8824
>                 URL: https://issues.apache.org/jira/browse/HDFS-8824
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: balancer & mover
>            Reporter: Tsz Wo Nicholas Sze
>            Assignee: Tsz Wo Nicholas Sze
>             Fix For: 2.8.0, 2.7.4, 3.0.0-alpha1
>
>         Attachments: h8824_20150727b.patch, h8824_20150811b.patch
>
>
> Balancer gets datanode block lists from NN and then move the blocks in order to balance
the cluster.  It should not use the blocks with small size since moving the small blocks generates
a lot of overhead and the small blocks do not help balancing the cluster much.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message