hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo Nicholas Sze (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9303) Balancer slowly with too many small file blocks
Date Thu, 29 Oct 2015 02:13:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14979663#comment-14979663
] 

Tsz Wo Nicholas Sze commented on HDFS-9303:
-------------------------------------------

Hi Lin, thanks.  We recently have quite a few improvement on Balancer; see the subtasks in
HDFS-8825.  For performance improvement, we have HDFS-8818 (one of the subtasks).  Since these
improvements only changes Balancer, you may apply the patches yourself and try them on your
cluster without upgrading the cluster.  I believe you could find more further improvements.
 I look forward to see your contribution!

> Balancer slowly with too many small file blocks
> -----------------------------------------------
>
>                 Key: HDFS-9303
>                 URL: https://issues.apache.org/jira/browse/HDFS-9303
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer & mover
>    Affects Versions: 2.7.1
>            Reporter: Lin Yiqun
>            Assignee: Lin Yiqun
>         Attachments: HDFS-9303.001.patch, HDFS-9303.002.patch
>
>
> In the recent hadoop release versions I found that balance operation is always so slowly,
even though I upgrade the version.When I analyse balancer log, I found that in every balance
iteration,it use only 4 to 5 minutes, and is a short time.And The most important is that the
most of being moving blocks is small blocks ,and the size is smaller than 1M.And this is a
main reason of the low effective of balance operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message