hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: IP based cluster - address space issues
Date Tue, 20 May 2008 20:58:08 GMT

Changing ip address alone should not affect HDFS.. especially if you 
restart it. You might be hitting some other "meta" issues.. check the 
datanode and namenode logs.

You should not have to lose any data or re-replicate it.


Jason Venner wrote:
> There is probably a better way than this rather brute force destructive 
> method.  If this fails, your hdfs is probably not recoverable so use at 
> your own risk.
>   1. return all the nodes to the original ip addresses
>   2. recursively set your replication count to higher than 1 and wait
>      for the replication updates to finish
>   3. turn hdfs off on REPCOUNT -1 machines in your cluster by running
>      bin/hadoop-damon.sh stop datanode on each of these machines.
>   4. put the old ip address in the exclude list on the master
>   5. change the ip addresses on the hdfs disabled machines to the new
>      ip addresses
>   6. remove or move away the contents of the dfs directory on the
>      machines (the hadoop.tmp.dir property)
>   7. add each of the machines back into the cluster by
>      bin/hadoop-damon.sh start datanode
>   8. wait until the replications re-balance the cluster either by
>      running the setrep -R command or by using the balancer.sh command
>   9. repeat 3-8 until all of the machines are on the new ip address
>  10. switch over the namenode
> pvvpr wrote:
>> Hello,
>>   We have a cluster that we initially configured using IP addresses
>> instead of hostnames (i.e., all namenode, datanode references are
>> mentioned as IP addresses rather than hostnames. We did this with an
>> intuition that the cluster might work faster). However, our entire IP
>> address subnet is now changing in our center due to reasons beyond our
>> control.
>>   When we changed the machine IP address to the new subnet, the datanodes
>> refuse to start. How can I have a clean start without messing my old
>> data?
>> thanks in advance for help.
>> - Prasad Pingali,
>> LTRC,
>> IIIT, Hyderabad.

View raw message