hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Re: What is the difference between "hbase.server.thread.wakefrequency" and "hbase.hregion.majorcompaction"?
Date Fri, 22 Aug 2014 04:28:16 GMT
Let's look at the code of MemStoreFlusher#FlushHandler#run() :

          fqe = flushQueue.poll(threadWakeFrequency, TimeUnit.MILLISECONDS);

After FlushQueueEntry is retrieved, flushRegion(fre) is called which calls
this method:

  private boolean flushRegion(final HRegion region, final boolean
emergencyFlush) {

where whether to trigger compaction is checked:

      boolean shouldCompact = region.flushcache().isCompactionNeeded();


On the other hand, major compaction decision is made by
RatioBasedCompactionPolicy#isMajorCompaction() which considers
hbase.hregion.majorcompaction for time based major compaction - see
getNextMajorCompactTime()


Cheers


On Thu, Aug 21, 2014 at 6:21 PM, chenwenhui <c77_cn@163.com> wrote:

> Thanks.
> Maybe i can not present my question clearly.
> I indeed understand the mean of the combination
> condition("hbase.server.thread.wakefrequency" and
> "server.compactchecker.interval.multiplier"),  i also know that its product
> is the period that check whether trigger compaction.
> For another combination condition("hbase.hregion.majorcompaction(7 days by
> default)" and "hbase.hregion.majorcompaction.jitter"), i understand it is
> also a condition to trigger compaction. right? if so, what is the relation
> between the two combination condition?
>
>
>
>
>
>
>
>
> At 2014-08-21 09:54:51, "Ted Yu" <yuzhihong@gmail.com> wrote:
> >See explanation for the following parameter (from hbase refguide)
> >
> >hbase.server.compactchecker.interval.multiplier
> >The number that determines how often we scan to see if compaction is
> >necessary. Normally, compactions are done after some events (such as
> >memstore flush), but if region     didn't receive a lot of writes for some
> >time, or due to different compaction policies, it may be necessary to
> check
> >it periodically. The interval between checks is hbase.
> >server.compactchecker.interval.multiplier multiplied by
> >hbase.server.thread.wakefrequency.
> >Default. 1000
> >Cheers
> >
> >
> >On Thu, Aug 21, 2014 at 3:01 AM, chenwenhui <c77_cn@163.com> wrote:
> >
> >> From the book named "Hbase: the Definitive Guide", i know that there is
> a
> >> condition to trigger major compaction, i.e. a thread controlled by
> >> "hbase.server.thread.wakefrequency(10 seconds by default)" and
> >> "hbase.server.thread.wakefrequency.multiplier" checks periodically
> whether
> >> major compaction.
> >> But i also read that the other conditon, both
> >> "hbase.hregion.majorcompaction(7 days by default)" and
> >> "hbase.hregion.majorcompaction.jitter", can also trigger major
> compaction.
> >> As my understanding, because of the former, the latter condition will
> >> never execute.
> >> Maybe i mis-undertand the two conditions, anyone can explain the
> >> difference between the two condtions? Thanks!
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message