hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randy Fox <r...@connexity.com>
Subject Re: Minor compacts seem blocked on major compacts
Date Wed, 07 Oct 2015 18:07:54 GMT
Looking at the source I don’t see how a manual major compact could get in the large compaction
pool:
If selectNow Is True the pool selector has to be shortCompactions.  From the logs it sure
seems that selectNow has to be true, else it would have logged “system” instead of "org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7f236eb8;”


Sure seems like an hbase bug.  

Am I missing something?

Thanks,

Randy

  /**
   * @param r HRegion store belongs to
   * @param s Store to request compaction on
   * @param why Why compaction requested -- used in debug messages
   * @param priority override the default priority (NO_PRIORITY == decide)
   * @param request custom compaction request. Can be <tt>null</tt> in which case
a simple
   *          compaction will be used.
   */
  private synchronized CompactionRequest requestCompactionInternal(final HRegion r, final
Store s,
      final String why, int priority, CompactionRequest request, boolean selectNow)
          throws IOException {
    if (this.server.isStopped()
        || (r.getTableDesc() != null && !r.getTableDesc().isCompactionEnabled()))
{
      return null;
    }

    CompactionContext compaction = null;
    if (selectNow) {
      compaction = selectCompaction(r, s, priority, request);
      if (compaction == null) return null; // message logged inside
    }

    // We assume that most compactions are small. So, put system compactions into small
    // pool; we will do selection there, and move to large pool if necessary.
    long size = selectNow ? compaction.getRequest().getSize() : 0;
    ThreadPoolExecutor pool = (!selectNow && s.throttleCompaction(size))
      ? longCompactions : shortCompactions;
    pool.execute(new CompactionRunner(s, r, compaction, pool));
    if (LOG.isDebugEnabled()) {
      String type = (pool == shortCompactions) ? "Small " : "Large ";
      LOG.debug(type + "Compaction requested: " + (selectNow ? compaction.toString() : "system")
          + (why != null && !why.isEmpty() ? "; Because: " + why : "") + "; " + this);
    }
    return selectNow ? compaction.getRequest() : null;
  }





On 10/6/15, 9:58 PM, "Randy Fox" <rfox@connexity.com> wrote:

>Sorry, I do not follow. I thought there was one thread per pool, one for large and one
for small.  Thus my major compact should have been in the large pool and not blocking the
small pool.  That is how it worked in 0.94.
>From the logs it demoted my compact to the small pool.  
>
>-randy
>
>
>
>On 10/6/15, 4:32 PM, "Vladimir Rodionov" <vladrodionov@gmail.com> wrote:
>
>>>> If I read this correct the size is 4.8G and the throttle is 2.5G so it
>>should have been put into the Large compaction pool.
>>
>>You answered your question yourself. Minor compaction in the same pool (1
>>thread default) will be waiting until major is finished.
>>
>>-Vlad
>>
>>
>>On Tue, Oct 6, 2015 at 3:59 PM, Randy Fox <rfox@connexity.com> wrote:
>>
>>> 2015-10-06 14:50:35,644 INFO org.apache.hadoop.hbase.regionserver.HStore:
>>> Starting compaction of 4 file(s) in L of PROD_audience4,\x00
>>> \xB6\x0B\xA7,\x186,1443751119137.26f321d7b240c85a9350a95f6c288e49. into
>>> tmpdir=hdfs://woz/hbase/data/default/PROD_audience4/26f321d7b240c85a9350a95f6c288e49/.tmp,
>>> totalSize=4.8 G
>>> 2015-10-06 14:50:35,646 DEBUG
>>> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Small Compaction
>>> requested:
>>> org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7f236eb8;
>>> Because: User-triggered major compaction; compaction_queue=(0:0),
>>> split_queue=0, merge_queue=0
>>>
>>>
>>> If I read this correct the size is 4.8G and the throttle is 2.5G so it
>>> should have been put into the Large compaction pool.
>>>
>>>
>>>
>>>
>>>
>>> On 10/6/15, 3:07 PM, "Randy Fox" <rfox@connexity.com> wrote:
>>>
>>> >We just did a big leap forward from 0.94 to 1.0. We do our own major
>>> compacts manually every night. One of the first things we noticed is that
>>> when a major compact runs, no minors run.  The compaction queue grows and
>>> when the major compact finishes the minors then run.   I have not found any
>>> new knobs we should be setting.  Any ideas?
>>> >Our config is:
>>> >
>>> ><property>
>>> >  <name>hbase.hregion.majorcompaction</name>
>>> >  <value>0</value>
>>> >  <final>true</final>
>>> ></property>
>>> ><property>
>>> >    <name>hbase.hstore.compactionThreshold</name>
>>> >    <value>3</value>
>>> ></property>
>>> >
>>> ><property>
>>> >    <name>hbase.hstore.compaction.max</name>
>>> >    <value>9</value>
>>> ></property>
>>> ><property>
>>> >    <name>hbase.regionserver.thread.compaction.throttle</name>
>>> >    <value>2684354560</value>
>>> ></property>
>>> >
>>> >
>>> >Thanks in advance,
>>> >
>>> >Randy
>>>
Mime
View raw message