hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: IP based cluster - address space issues
Date Tue, 20 May 2008 21:00:30 GMT
Jason Venner wrote:
> There is probably a better way than this rather brute force destructive 
> method.  If this fails, your hdfs is probably not recoverable so use at 
> your own risk.

Just to clarify, I strongly advise against the following method. It is 
better to find out and properly fix the actual problem. Changing ip 
address is supposed to work just fine.


>   1. return all the nodes to the original ip addresses
>   2. recursively set your replication count to higher than 1 and wait
>      for the replication updates to finish
>   3. turn hdfs off on REPCOUNT -1 machines in your cluster by running
>      bin/hadoop-damon.sh stop datanode on each of these machines.
>   4. put the old ip address in the exclude list on the master
>   5. change the ip addresses on the hdfs disabled machines to the new
>      ip addresses
>   6. remove or move away the contents of the dfs directory on the
>      machines (the hadoop.tmp.dir property)
>   7. add each of the machines back into the cluster by
>      bin/hadoop-damon.sh start datanode
>   8. wait until the replications re-balance the cluster either by
>      running the setrep -R command or by using the balancer.sh command
>   9. repeat 3-8 until all of the machines are on the new ip address
>  10. switch over the namenode
> pvvpr wrote:
>> Hello,
>>   We have a cluster that we initially configured using IP addresses
>> instead of hostnames (i.e., all namenode, datanode references are
>> mentioned as IP addresses rather than hostnames. We did this with an
>> intuition that the cluster might work faster). However, our entire IP
>> address subnet is now changing in our center due to reasons beyond our
>> control.
>>   When we changed the machine IP address to the new subnet, the datanodes
>> refuse to start. How can I have a clean start without messing my old
>> data?
>> thanks in advance for help.
>> - Prasad Pingali,
>> LTRC,
>> IIIT, Hyderabad.

View raw message