Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of lhofhansl@yahoo.com
 designates 72.30.239.198 as permitted sender)
References: 
 <CAEf3A-xGN7ygg1rTadtS_iTKtjCZ_1migmUFkgwOCEMRrdrFqg@mail.gmail.com>
	<1408080840.42288.YahooMailNeo@web140606.mail.bf1.yahoo.com>
 <CAEf3A-zgP0LpmpL7yaEk=qWjNBpci3Cg8HrbXFamM=po7GG7hQ@mail.gmail.com>
Message-ID: <1408125123.47575.YahooMailNeo@web140603.mail.bf1.yahoo.com>
Date: Fri, 15 Aug 2014 10:52:03 -0700
From: lars hofhansl <larsh@apache.org>
Reply-To: lars hofhansl <larsh@apache.org>
Subject: Re: A better way to migrate the whole cluster?
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
In-Reply-To: 
 <CAEf3A-zgP0LpmpL7yaEk=qWjNBpci3Cg8HrbXFamM=po7GG7hQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="-1172831624-1272786962-1408125123=:47575"

---1172831624-1272786962-1408125123=:47575
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Try to add these: =0A=0A=0A-Dhbase.client.scanner.caching=3D100=0A-Dmapred.=
map.tasks.speculative.execution=3Dfalse=0A=0A=0AAlso, as others pointed out=
, what's the bandwidth between the two clusters?=0A=0A=0A=0A_______________=
_________________=0A From: tobe <tobeg3oogle@gmail.com>=0ATo: dev@hbase.apa=
che.org; lars hofhansl <larsh@apache.org> =0ASent: Thursday, August 14, 201=
4 11:24 PM=0ASubject: Re: A better way to migrate the whole cluster?=0A =0A=
=0AThank @lars.=0A=0AWe're using HBase 0.94.11 and follow the instruction t=
o run `./bin/hbase=0Aorg.apache.hadoop.hbase.mapreduce.CopyTable --peer.adr=
=3Dhbase://cluster_name=0Atable_name`. We have namespace service to find th=
e ZooKeeper with=0A"hbase://cluster_name". And the job ran on a shared yarn=
 cluster.=0A=0AThe performance is affected by many factors, but we haven't =
found out the=0Areason. It would be great to see your suggestions.=0A=0A=0A=
=0AOn Fri, Aug 15, 2014 at 1:34 PM, lars hofhansl <larsh@apache.org> wrote:=
=0A=0A> What version of HBase? How are you running CopyTable? A day for 1.8=
T is=0A> not what we would expect.=0A> You can definitely take a snapshot a=
nd then export the snapshot to another=0A> cluster, which will move the act=
ual files; but CopyTable should not be so=0A> slow.=0A>=0A>=0A> -- Lars=0A>=
=0A>=0A>=0A> ________________________________=0A>=A0 From: tobe <tobeg3oogl=
e@gmail.com>=0A> To: "user@hbase.apache.org" <user@hbase.apache.org>=0A> Cc=
: dev@hbase.apache.org=0A> Sent: Thursday, August 14, 2014 8:18 PM=0A> Subj=
ect: A better way to migrate the whole cluster?=0A>=0A>=0A> Sometimes our u=
sers want to upgrade their servers or move to a new=0A> datacenter, then we=
 have to migrate the data from HBase. Currently we=0A> enable the replicati=
on from the old cluster to the new cluster, and run=0A> CopyTable to move t=
he older data.=0A>=0A> It's a little inefficient. It takes more than one da=
y to migrate 1.8T data=0A> and more time to verify. Can we have a better wa=
y to do that, like snapshot=0A> or purely HDFS files?=0A>=0A> And what's th=
e best practise or your valuable experience?=0A>
---1172831624-1272786962-1408125123=:47575--