hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeremy Carroll <phobos...@gmail.com>
Subject Re: When should we trigger a major compaction?
Date Fri, 21 Feb 2014 17:17:44 GMT
When the store file size starts to get very large, major compaction will shrink the HFile size
down by removing tombstones and expired records.  If latency in requests is within your SLA,
store file size is not gigantic. You would not have to do this. 

IMHO minor compactions is the best tool to enforce the max number of store files read in a
request. 

Sent from my iPhone

> On Feb 21, 2014, at 6:26 AM, Ramon Wang <ramon@appannie.com> wrote:
> 
> Guys, I'm clear about what compaction is doing there, i just want to know
> are there any indicators or indexes can help us to decide whether to start
> a major compaction manually?
> 
> As there will be a single StoreFile per Store after a major compaction, so
> the num of StoreFile maybe a indexes, but i'm not sure about this.
> 
> Thanks
> Ramon
> 
> 
>> On Fri, Feb 21, 2014 at 9:00 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> 
>> Ramon:
>> See http://hbase.apache.org/book.html#compaction for description on
>> compaction file selection.
>> 
>> Cheers
>> 
>> 
>>> On Fri, Feb 21, 2014 at 4:38 AM, yonghu <yongyong313@gmail.com> wrote:
>>> 
>>> Before you want to trigger major compaction, let's first explain why do
>> we
>>> need major compaction. The major compaction will cause
>>> 1. delete the data which is masked by tombstone;
>>> 2. delete the data which has expired ttl;
>>> 3. compact several small hfiles into a single larger one.
>>> 
>>> I didn't quite understand what do you mean by "use num of storefiles",
>> why
>>> this?
>>> 
>>> 
>>>> On Fri, Feb 21, 2014 at 10:50 AM, Ramon Wang <ramon@appannie.com> wrote:
>>>> 
>>>> Hi Guys
>>>> 
>>>> We disabled the automatically major compaction setting in our HBase
>>>> cluster, so we want to something which can tell us when should we do
>>> major
>>>> compaction, we were thinking to use "num of storefiles", but we cannot
>>> find
>>>> any APIs or any JMX settings which can be used for it, please share us
>>> some
>>>> ideas on how to do it, thanks in advance.
>>>> 
>>>> Cheers
>>>> Ramon
>> 

Mime
View raw message