hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: hbase migration
Date Sat, 12 Feb 2011 05:41:16 GMT
This is Xavier's blog on moving lots of hbase data:
http://blog.mozilla.com/data/2011/02/04/migrating-hbase-in-the-trenches/
 They had trouble using straight distcp.

Do you want to move a table at at time.  If so, you'll need something
to add the regions back up into .META. when they arrive at cluster B.
bin/add_table.rb won't assign the regions in 0.90.  You could force
enable of the copied table.  That should do it.

Can you take some down time?  You need to flush the 0.20.x cluster
before you do final copy to get memory content.

St.Ack

On Fri, Feb 11, 2011 at 3:32 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> I don't think replication is in 0.20.6
>
> Can we do this:
> create table with same schema on cluster B.
> distcp copy over /hbase/tableX from cluster A to B ?
>
> On Fri, Feb 11, 2011 at 3:27 PM, Jeff Whiting <jeffw@qualtrics.com> wrote:
>
>> The way we've moved data from cluster A to cluster B is:
>> 1. Turn on replication from A -> B
>> 2. Run map reduce export on A and map reduce import on B
>>
>> The time stamp functionality of hbase makes it so the import to B wont
>> overwrite any of the newer replicated data.  Once the import is complete we
>> simply point the clients at cluster B and we're done.
>>
>> However I'm not sure if the replication functionality is in 0.20.6.
>>
>> ~Jeff
>>
>>
>> On 2/11/2011 3:48 PM, Ted Yu wrote:
>>
>>> Are you referring to https://issues.apache.org/jira/browse/HBASE-3451 ?
>>>
>>> Our clusters are in same DC but not on same hdfs.
>>>
>>> Our issue is that the table to be transferred doesn't exist on destination
>>> cluster.
>>>
>>> Thanks
>>>
>>> On Fri, Feb 11, 2011 at 2:41 PM, Stack<stack@duboce.net>  wrote:
>>>
>>>  On Fri, Feb 11, 2011 at 2:32 PM, Ted Yu<yuzhihong@gmail.com>  wrote:
>>>>
>>>>> QA can use export/import tool
>>>>>
>>>>> I wonder if there is a faster way of transferring whole table across
>>>>> clusters.
>>>>>
>>>>>  You saw the recent mozilla posting by xavier about wrangling
>>>> x-datacenter copying?  If not, check it out.
>>>>
>>>> Are your clusters in same DC or even on same HDFS?
>>>>
>>>> St.Ack
>>>>
>>>>
>> --
>> Jeff Whiting
>> Qualtrics Senior Software Engineer
>> jeffw@qualtrics.com
>>
>>
>

Mime
View raw message