hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhou Shuaifeng <zhoushuaif...@huawei.com>
Subject 答复: Can region be merged with others automatically when all data in the region has expired and removed ?
Date Wed, 09 Feb 2011 08:59:30 GMT
We have test a cluster which have more than 30,000 regions, max size of a region is 512MB.
At this situation, data no more growing, but remove some old data and insert new, and regions
will be more and more.
This occupies too much heapsize, and will be more if regions cannot be merged. And it takes
too long to make the table offline.

Zhou Shuaifeng(Frank)
HUAWEI TECHNOLOGIES CO.,LTD. 

-------------------------------------------------------------------------------------------------------------------------------------
This e-mail and its attachments contain confidential information from HUAWEI, which 
is intended only for the person or entity whose address is listed above. Any use of the 
information contained herein in any way (including, but not limited to, total or partial 
disclosure, reproduction, or dissemination) by persons other than the intended 
recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
by 
phone or email immediately and delete it!


-----邮件原件-----
发件人: Ryan Rawson [mailto:ryanobjc@gmail.com] 
发送时间: 2011年2月9日 14:41
收件人: dev@hbase.apache.org
主题: Re: Can region be merged with others automatically when all data in the region has
expired and removed ?

I'm curious, if you are expiring a lot of data, does your table grow?
If not, could you fit it in to a mysql instance instead?

As for pre-splitting tables, if you have a really large data set, how
would you manage this? One of our tables has 700 regions, and we didnt
pre-split. I didnt really know the distribution of keys before I
started inserting data, and I'd rather just let HBase do the right
thing.

-ryan

On Tue, Feb 8, 2011 at 10:21 PM, Ted Dunning <tdunning@maprtech.com> wrote:
> Online merge is a bit dangerous.  Lots of applications require that the
> table be set up pre-split.  This is probably more common than the need for
> merging.
>
> Having such a pre-split table collapse before it is full would be a
> disaster.
>
> It should be pretty easy to script taking a few regions off-line and then
> nuking them.
>
> 2011/2/8 Jean-Daniel Cryans <jdcryans@apache.org>
>
>> An automatic online merge feature would be a nice contribution too.
>>
>



Mime
View raw message