hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Venner <ja...@attributor.com>
Subject Re: IP based cluster - address space issues
Date Tue, 20 May 2008 20:51:00 GMT
There is probably a better way than this rather brute force destructive 
method.  If this fails, your hdfs is probably not recoverable so use at 
your own risk.

   1. return all the nodes to the original ip addresses
   2. recursively set your replication count to higher than 1 and wait
      for the replication updates to finish
   3. turn hdfs off on REPCOUNT -1 machines in your cluster by running
      bin/hadoop-damon.sh stop datanode on each of these machines.
   4. put the old ip address in the exclude list on the master
   5. change the ip addresses on the hdfs disabled machines to the new
      ip addresses
   6. remove or move away the contents of the dfs directory on the
      machines (the hadoop.tmp.dir property)
   7. add each of the machines back into the cluster by
      bin/hadoop-damon.sh start datanode
   8. wait until the replications re-balance the cluster either by
      running the setrep -R command or by using the balancer.sh command
   9. repeat 3-8 until all of the machines are on the new ip address
  10. switch over the namenode

pvvpr wrote:
> Hello,
>   We have a cluster that we initially configured using IP addresses
> instead of hostnames (i.e., all namenode, datanode references are
> mentioned as IP addresses rather than hostnames. We did this with an
> intuition that the cluster might work faster). However, our entire IP
> address subnet is now changing in our center due to reasons beyond our
> control.
>   When we changed the machine IP address to the new subnet, the datanodes
> refuse to start. How can I have a clean start without messing my old
> data?
> thanks in advance for help.
> - Prasad Pingali,
> IIIT, Hyderabad.
Jason Venner
Attributor - Program the Web <http://www.attributor.com/>
Attributor is hiring Hadoop Wranglers and coding wizards, contact if 

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message