hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Liu Shaohui (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-8131) Implement a space blance block placement policy
Date Mon, 13 Apr 2015 04:14:12 GMT
Liu Shaohui created HDFS-8131:

             Summary: Implement a space blance block placement policy
                 Key: HDFS-8131
                 URL: https://issues.apache.org/jira/browse/HDFS-8131
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: Liu Shaohui
            Assignee: Liu Shaohui
            Priority: Minor

The default block placement policy will choose datanodes for new blocks randomly, which will
result in unbalanced space used percent among datanodes after an cluster expansion. The old
datanodes always are in high used percent of space and new added ones are in low percent.

Through we can used the external balance tool to balance the space used rate, it will cost
extra network IO and it's not easy to control the balance speed.

An easy solution is to implement an balanced block placement policy which will choose low
used percent datanodes for new blocks with a little high possibility. In a not long term,
the used percent of datanodes will trend to be balanced.

Suggestions and discussions are welcomed. Thanks

This message was sent by Atlassian JIRA

View raw message