hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benoy Antony (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-6441) Add ability to Ignore few datanodes while balancing
Date Thu, 22 May 2014 09:55:01 GMT

    [ https://issues.apache.org/jira/browse/HDFS-6441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14005796#comment-14005796
] 

Benoy Antony commented on HDFS-6441:
------------------------------------

Thanks for the pointer, [~carp84]. After reviewing HDFS-6010's requirements, I have updated
the patch to accept either an exclude list of datanodes or an include list of datanodes. This
required only a minor change to the patch. 

I have changed the the parameter to "-datanodes" .
Also added two flags - "-exclude" and "-include"

The exclude list can be specified as  
{panel}
run-balancer.sh  -threshold 10 -exclude -datanodes <hosts-file>
{panel}
With this, the datanodes in the hosts-file will not be used while balancing.

The include list can be specified as 
{panel}
run-balancer.sh  -threshold 10 -include -datanodes <hosts-file>
{panel}
With this, only the datanodes in the hosts-file will be used for balancing. 

When there are many hosts, it will be better to keep the hosts in a file rather than pass
it via command line.


> Add ability to Ignore few datanodes while balancing
> ---------------------------------------------------
>
>                 Key: HDFS-6441
>                 URL: https://issues.apache.org/jira/browse/HDFS-6441
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer
>    Affects Versions: 2.4.0
>            Reporter: Benoy Antony
>            Assignee: Benoy Antony
>         Attachments: HDFS-6441.patch
>
>
> In some use cases, it is desirable to ignore a few data nodes  while balancing. The administrator
should be able to specify a list of data nodes in a file similar to the hosts file and the
balancer should ignore these data nodes while balancing so that no blocks are added/removed
on these nodes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message