hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arthur Chan <arthur.hk.c...@gmail.com>
Subject Re: Hadoop 2.6.0, How to add/remove node to/from running cluster
Date Sun, 05 Apr 2015 00:58:44 GMT
Hi,  Thanks.

For 3), about "dfs.hosts, dfs.hosts.exclude", how many configuration XML
files should be modified?

Could you advise if my following steps are correct?

=== My Hadoop is 2.6.0 ===
Step 1:
edit mapred-site.xml (ADD)


step 2:

edit hdfs-site.xml (ADD)


step 3:
edit /myhadoop/etc/hadoop/datanode-excludes, add the full names of all
nodes to it
edit /myhadoop/etc/hadoop/datanode-excludes, add the full name of the node
to be decommissioned

step 4:  run the following command

*hdfs dfsadmin –refreshNodes*

*hdfs mradmin –refreshNodes  *

Q1) It seems that for 2.6.0 (or even 2.4.1) the correct way should not
use "*hdfs
dfsadmin –refreshNodes", *
instead, should use "sbin/*distribute-exclude.sh"*

Q2) I tried *hdfs mradmin –refreshNodes, got * "Error: Could not find or
load main class mradmin"

Please advise the correct way to decommission a datanode completely (for
both DFS and Nodemanager)  in Hadoop 2.x

On Sun, Apr 5, 2015 at 12:53 AM, Akira AJISAKA <ajisakaa@oss.nttdata.co.jp>

> Hi Arthur,
> > 1) How to add a new datanode to a running Hadoop 2.6.0 cluster?
> Just starting a datanode is fine.
> The datanode will be added to the cluster automatically.
> > 2) How to rebalance the cluster after the new node is added?
> Please see http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/
> hadoop-hdfs/HdfsUserGuide.html#Balancer
> > 3) I could not find the section about how to decommission a datanode
> > from a running cluster
> Please see -refreshNodes section of this document.
> http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/
> hadoop-hdfs/HdfsUserGuide.html#Shell_Commands
> Regards,
> Akira

View raw message