Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of ryanobjc@gmail.com designates
 209.85.223.189 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=MEsNBXGskAEQZYJ/WpshbjLbhLHP6kh09Dv+Q/yMZGd1DoULm+uoUIT9lWISRaUDgH
         WfxB29pPfIvMRw64v0ApuxzXOsNCWwW6Cyr9CMMfqtZy5b02ni1TVBrLw+1zNFHLeqjR
         XU968x4jx5rB3XqN3DwBr3JqTfrUKtGpAEwhc=
MIME-Version: 1.0
In-Reply-To: <7c457ebe1002091902l707cb116g1079257ec7292105@mail.gmail.com>
References: <1265770076.28098.299.camel@puma>
	 <78568af11002091850p494d0815gef5f475f4d0c4479@mail.gmail.com>
	 <7c457ebe1002091902l707cb116g1079257ec7292105@mail.gmail.com>
Date: Tue, 9 Feb 2010 19:07:48 -0800
Message-ID: <78568af11002091907o9dd4f45rb265e9c6330ccb46@mail.gmail.com>
Subject: Re: HBase export/import
From: Ryan Rawson <ryanobjc@gmail.com>
To: hbase-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

You can also use distcp instead of the copyToLocal && scp &&
copyFromLocal chain you have there.

On Tue, Feb 9, 2010 at 7:02 PM, Dan Washusen <dan@reactive.org> wrote:
> +1
>
> I use this method when performance testing on different data sets. =A0I h=
ave
> several datasets to test on (varying sizes, etc). =A0 When I want to swit=
ch
> datasets I just shut down hbase and rename the /hbase directory...
>
> e.g. (assuming hbase is not running)
> hadoop/bin/hadoop fs -mv /hbase /hbase.small
> hadoop/bin/hadoop fs -mv /hbase.large /hbase
>
> When I want to move my data between clusters I use:
> hadoop/bin/hadoop fs -copyToLocal /hbase.large /tmp/hbase.large
> scp -r /tmp/hbase.large user@host:/tmp
> ssh user@host
> hadoop/bin/hadoop fs -put /tmp/hbase.large /hbase
>
>
> Very handy :)
>
>
> On 10 February 2010 13:50, Ryan Rawson <ryanobjc@gmail.com> wrote:
>
>> If you stop the source cluster then you can distcp the /hbase to the
>> other cluster. Done. A perfect copy.
>>
>> That is probably the most efficient/highest performing way.
>>
>> On Tue, Feb 9, 2010 at 6:47 PM, James Baldassari <james@dataxu.com> wrot=
e:
>> > Hi,
>> >
>> > I'm wondering if it's possible to export all data from one HBase clust=
er
>> > and import it into another. =A0We have a lot of data that we've import=
ed
>> > into our staging HBase environment, and rather than repeating the
>> > lengthy import process in our production environment we would prefer t=
o
>> > just copy all the data directly from HBase/HDFS in staging into
>> > production. =A0Is there an easy way to do this? =A0I know Hadoop has s=
ome
>> > distributed copy functionality, but I don't know if this will work wit=
h
>> > HBase. =A0The number of region servers and the replication factor will=
 be
>> > the same in the source and destination environments, but the
>> > hostnames/IPs will be different. =A0The production environment is
>> > completely empty right now, so we don't need to worry about overwritin=
g
>> > data.
>> >
>> > I came across these links while searching for information HBase
>> > export/import:
>> >
>> > http://issues.apache.org/jira/browse/HBASE-897
>> > http://issues.apache.org/jira/browse/HBASE-1684
>> >
>> http://hadoop.apache.org/hbase/docs/current/api/org/apache/hadoop/hbase/=
mapreduce/Export.html
>> >
>> > Has anyone used these tools? =A0Is there a better way?
>> >
>> > Thanks,
>> > James
>> >
>> >
>> >
>>
>