Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: error (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CAHtRq2cPx2pjY0DnToRT2V0BLxHes7u0xtQ+JLw6g7tb8g7bGg@mail.gmail.com>
References: 
 <CAHtRq2cPx2pjY0DnToRT2V0BLxHes7u0xtQ+JLw6g7tb8g7bGg@mail.gmail.com>
From: Ted Dunning <tdunning@maprtech.com>
Date: Thu, 18 Apr 2013 02:23:10 +0000
Message-ID: 
 <CAND0qzvSkLZw9FkgCSKqts0jgc+eF3g9a4U1KurpmOmp2N4iAg@mail.gmail.com>
Subject: Re: Physically moving HDFS cluster to new
To: "common-user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=047d7b3a81fedcd39004da994998

--047d7b3a81fedcd39004da994998
Content-Type: text/plain; charset=ISO-8859-1

It may or may not help you in your current distress, but MapR's
distribution could handle this pretty easily.

One method is direct distcp between clusters, but you could also use MapR's
mirroring capabilities to migrate data.

You can also carry a MapR cluster, change the IP addresses and relight the
cluster without data loss.  You can also move disks (respecting RAID-0 disk
groups, of course) from machine to machine within a cluster and have them
wake up with all file and directory meta-data intact.

Furthermore, you can lose any two machines in a cluster and are guaranteed
to be able to reconstruct the cluster.  Even if you lose all three replicas
of *any* of the data or meta-data in the cluster, you can *still*
reconstruct any data volumes for which at least one copy survives.  The the
lost data volumes come back at a later time, you will also be able
resurrect the data correctly.

None of this is true for any of the other major Hadoop distributions.

Let me know if you want to try this out.


On Wed, Apr 17, 2013 at 5:20 PM, Tom Brown <tombrown52@gmail.com> wrote:

> We have a situation where we want to physically move our small (4 node)
> cluster from one data center to another. As part of this move, each node
> will receive both a new FQN and a new IP address. As I understand it, HDFS
> is somehow tied to the the FQN or IP address, and changing them causes data
> loss.
>
> Is there any supported method of moving a cluster this way?
>
> Thanks in advance!
>
> --Tom
>

--047d7b3a81fedcd39004da994998
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><br><div style>It may or may not help you in your current =
distress, but MapR&#39;s distribution could handle this pretty easily.</div=
><div style><br></div><div style>One method is direct distcp between cluste=
rs, but you could also use MapR&#39;s mirroring capabilities to migrate dat=
a.</div>

<div style><br></div><div style>You can also carry a MapR cluster, change t=
he IP addresses and relight the cluster without data loss. =A0You can also =
move disks (respecting RAID-0 disk groups, of course) from machine to machi=
ne within a cluster and have them wake up with all file and directory meta-=
data intact.</div>

<div style><br></div><div style>Furthermore, you can lose any two machines =
in a cluster and are guaranteed to be able to reconstruct the cluster. =A0E=
ven if you lose all three replicas of *any* of the data or meta-data in the=
 cluster, you can *still* reconstruct any data volumes for which at least o=
ne copy survives. =A0The the lost data volumes come back at a later time, y=
ou will also be able resurrect the data correctly.</div>

<div style><br></div><div style>None of this is true for any of the other m=
ajor Hadoop distributions.</div><div style><br></div><div style>Let me know=
 if you want to try this out.</div><div style><br></div><div style><br>

</div><div style><br></div></div><div class=3D"gmail_extra"><br><br><div cl=
ass=3D"gmail_quote">On Wed, Apr 17, 2013 at 5:20 PM, Tom Brown <span dir=3D=
"ltr">&lt;<a href=3D"mailto:tombrown52@gmail.com" target=3D"_blank">tombrow=
n52@gmail.com</a>&gt;</span> wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">We have a situation where w=
e want to physically move our small (4 node) cluster from one data center t=
o another. As part of this move, each node will receive both a new FQN and =
a new IP address. As I understand it, HDFS is somehow tied to the the FQN o=
r IP address, and changing them causes data loss.<div>


<br></div><div><div>Is there any supported method of moving a cluster this =
way?</div></div><div><br></div><div>Thanks in advance!</div><span class=3D"=
HOEnZb"><font color=3D"#888888"><div><br></div><div>--Tom</div></font></spa=
n></div>


</blockquote></div><br></div>

--047d7b3a81fedcd39004da994998--