hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rajiv Chittajallu <raj...@yahoo-inc.com>
Subject Re: Physically moving HDFS cluster to new
Date Thu, 18 Apr 2013 03:27:02 GMT


On 4/17/13 7:23 PM, "Ted Dunning" <tdunning@maprtech.com> wrote:

>
>It may or may not help you in your current distress, but MapR's
>distribution could handle this pretty easily.
>
>
>One method is direct distcp between clusters, but you could also use
>MapR's mirroring capabilities to migrate data.
>
>
>You can also carry a MapR cluster, change the IP addresses and relight
>the cluster without data loss.  You can also move disks (respecting
>RAID-0 disk groups, of course) from machine to machine within a cluster
>and have them wake up with all file
> and directory meta-data intact.
>
>
>Furthermore, you can lose any two machines in a cluster and are
>guaranteed to be able to reconstruct the cluster.  Even if you lose all
>three replicas of *any* of the data or meta-data in the cluster, you can
>*still* reconstruct any data volumes
> for which at least one copy survives.  The the lost data volumes come
>back at a later time, you will also be able resurrect the data correctly.
>
>
>None of this is true for any of the other major Hadoop distributions.


This is not correct. In HDFS, As long as the fsimage and copy of a data
block are kept intact you can do any changes to the nodes.


>
>Let me know if you want to try this out.
>
>
>On Wed, Apr 17, 2013 at 5:20 PM, Tom Brown
><tombrown52@gmail.com> wrote:
>
>We have a situation where we want to physically move our small (4 node)
>cluster from one data center to another. As part of this move, each node
>will receive both a new FQN and a new IP address. As I understand it,
>HDFS is somehow tied to the
> the FQN or IP address, and changing them causes data loss.
>
>
>Is there any supported method of moving a cluster this way?
>
>
>
>Thanks in advance!
>
>
>--Tom
>
>
>
>
>
>


Mime
View raw message