Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of yongyong313@gmail.com
 designates 209.85.210.41 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <CB470B5B.2A682%nspiegelberg@fb.com>
References: 
 <CABALG=Si4gm3ZYsuUPgNTuN0Xd0n_jiWizz-euc0ZR71KYRjdA@mail.gmail.com>
	<CB470B5B.2A682%nspiegelberg@fb.com>
Date: Thu, 26 Jan 2012 22:00:19 +0100
Message-ID: 
 <CABALG=T0ORL1n=wUhZ8+LYwDhBAMAVBGj0DcOSytbHDRxckvhw@mail.gmail.com>
Subject: Re: the occasion of the major compact?
From: yonghu <yongyong313@gmail.com>
To: user@hbase.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

Nicolas,

In my use case, I want to extract the deleted data. Hence, if I
disable the major compaction, I can prevent the hbase to actually
delete the data. After extracting the deleted data, I can issue major
compact by myself.

Regards

Yong

On Thu, Jan 26, 2012 at 8:02 PM, Nicolas Spiegelberg
<nspiegelberg@fb.com> wrote:
> Yong,
>
> Can you please explain why you want to disable major compactions? =A0What
> are the problems that you're currently seeing or what are you worried wil=
l
> happen if a major compaction is allowed to occur? =A0Right now, there are
> only an extremely small subset of cases where you must explicitly disable
> compactions. =A0These use cases I know of are very complicated and requir=
e
> building StoreFile analysis tools underneath HBase, that I'm pretty sure
> you're not needing this.
>
> Please also read my follow up commentary to explaining major compaction
> logic:
> http://search-hadoop.com/m/JR9sK1xnbj21
> http://search-hadoop.com/m/X7W7q1xnbj21
>
>
> The vast majority of users need features completely unrelated to
> compactions. =A0The compaction algorithm is an easy target to worry about=
.
>
>
> On 1/26/12 7:06 AM, "yonghu" <yongyong313@gmail.com> wrote:
>
>>Hello Mikael,
>>
>>I think disabling the major compaction in the timed and client-issued
>>situation is not a problem. The problem is the size-based. From the
>>mailing list, it only talks about the situation of minor compaction
>>not major compaction, if I understand right. So, I want to know if
>>someone can tell me how to close the major compaction in size-based
>>situation.
>>
>>Thanks
>>
>>Yong
>>I saw the description which indicating the size of store file can also
>>trigger major compaction.
>>
>>On Thu, Jan 26, 2012 at 3:54 PM, Mikael Sitruk <mikael.sitruk@gmail.com>
>>wrote:
>>> Yong hi
>>>
>>> As far as i know setting =A0hbase.hregion.majorcompaction to 0 will
>>>disable
>>> the time based trigger only.
>>> Client are always able to invoke the major compact, no matter what is
>>>the
>>> value of the hbase.hregion.majorcompaction.
>>>
>>> Perhaps client invocation of compaction can me disabled with the
>>>security
>>> package.
>>>
>>> Anyway i'm digging into 0.92, I hope to get those insight soon.
>>>
>>> Mikael.S
>>>
>>> On Thu, Jan 26, 2012 at 4:39 PM, yonghu <yongyong313@gmail.com> wrote:
>>>
>>>> Thanks for your response.
>>>>
>>>> I knew that major compact can be triggered based on client, time and
>>>> size. In my situation, I have to close the functionality of major
>>>> compact. So, if I set the =8Chbase.hregion.majorcompaction=B9 into 0, =
it
>>>> will close all the three situations or I have to set it separately for
>>>> each case. BTW, my hbase version is 0.92.
>>>>
>>>> Thanks!
>>>>
>>>> Yong
>>>>
>>>> On Thu, Jan 26, 2012 at 3:09 PM, Mikael Sitruk
>>>><mikael.sitruk@gmail.com>
>>>> wrote:
>>>> > look at the thread http://search-hadoop.com/m/GHUWQ1xnbj21, it
>>>>explain a
>>>> > lot on major compaction and enhancement over versions
>>>> >
>>>> > Mikael.S
>>>> >
>>>> >
>>>> > On Thu, Jan 26, 2012 at 3:51 PM, Damien Hardy <dhardy@figarocms.fr>
>>>> wrote:
>>>> >
>>>> >> Le 26/01/2012 14:43, yonghu a =E9crit :
>>>> >> > Hello,
>>>> >> >
>>>> >> > I read this blog http://outerthought.org/blog/465-ot.html. It
>>>> mentions
>>>> >> > that every 24 hours the major compaction will occur. My question
>>>>is
>>>> >> > that if there are any other conditions which can trigger major
>>>> >> > compaction happening? For example, when the size of store file
>>>>reaches
>>>> >> > the threshold (I think this will cause minor compaction or region
>>>>file
>>>> >> > split, not major compaction, but not quite sure).
>>>> >> >
>>>> >> > Thanks!
>>>> >> >
>>>> >> > Yong
>>>> >>
>>>> >> Hello,
>>>> >> I think when there is massive delete on the table or change table
>>>> >> attribute like TTL (that is susseptible of remove a lot of
>>>> >> versions/rows) or COMPRESSION wich gain a lot of disk space on each
>>>> region.
>>>> >>
>>>> >> Cheers,
>>>> >>
>>>> >> --
>>>> >> Damien
>>>> >>
>>>> >>
>>>> >
>>>> >
>>>> > --
>>>> > Mikael.S
>>>>
>>>
>>>
>>>
>>> --
>>> Mikael.S
>