Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 9 Dec 2014 21:17:12 +0000 (UTC)
From: "Vladimir Rodionov (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12760279.1418076033000.112920.1418159832842@Atlassian.JIRA>
In-Reply-To: <JIRA.12760279.1418076033000@Atlassian.JIRA>
References: <JIRA.12760279.1418076033000@Atlassian.JIRA>
 <JIRA.12760279.1418076033553@arcas>
Subject: [jira] [Commented] (HBASE-12657) The Region is not being split and
 far exceeds the desired maximum size.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-12657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14240081#comment-14240081 ] 

Vladimir Rodionov commented on HBASE-12657:
-------------------------------------------

There is a difference between 0.94 and 0.98+ in how we remove excess files:

0.94 - from the head

{code}
      if(majorcompaction) {
        if (compactSelection.getFilesToCompact().size() > this.maxFilesToCompact) {
          LOG.debug("Warning, compacting more than " + this.maxFilesToCompact +
            " files, probably because of a user-requested major compaction");
          if(priority != PRIORITY_USER) {
            LOG.error("Compacting more than max files on a non user-requested compaction");
          }
        }
      } else if (compactSelection.getFilesToCompact().size() > this.maxFilesToCompact) {
        // all files included in this compaction, up to max
        int pastMax = compactSelection.getFilesToCompact().size() - this.maxFilesToCompact;
        compactSelection.getFilesToCompact().subList(0, pastMax).clear();
      }
{code}

trunk & 0.98 - from the tail

{code}
      if (isMajorCompaction && isUserCompaction) {
        LOG.debug("Warning, compacting more than " + comConf.getMaxFilesToCompact() +
            " files because of a user-requested major compaction");
      } else {
        LOG.debug("Too many admissible files. Excluding " + excess
          + " files from compaction candidates");
        candidates.subList(comConf.getMaxFilesToCompact(), candidates.size()).clear();
      }
{code}


> The Region is not being split and far exceeds the desired maximum size.
> -----------------------------------------------------------------------
>
>                 Key: HBASE-12657
>                 URL: https://issues.apache.org/jira/browse/HBASE-12657
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>    Affects Versions: 0.98.8, 0.94.25, 0.99.2
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>             Fix For: 1.0.0, 2.0.0, 0.94.26, 0.98.9
>
>
> We are seeing this behavior when creating indexes in one of our environment.
> When an index is being created, most of the "requests" go into a single region.  The amount of time to create an index seems to take longer than usual and it can take days for the regions to compact and split after the index is created.
> Here is a du of the HBase index table:
> {code}
> -bash-4.1$ sudo -su hdfs hadoop fs -du /hbase/43681
> 705          /hbase/43681/.tableinfo.0000000001
> 0            /hbase/43681/.tmp
> 27981697293  /hbase/43681/0492e22092e21d35fca8e779b21ec797
> 539687093    /hbase/43681/832298c4e975fc47210feb6bac3d2f71
> 560660531    /hbase/43681/be9bdb3bdf9365afe5fe90db4247d82c
> 7081938297   /hbase/43681/cd440e524f96fbe0719b2fe969848560
> 6297860287   /hbase/43681/dc893a2d8daa08c689dc69e6bb2c5b50
> 7189607722   /hbase/43681/ffbceaea5e2f142dbe6cd4cbeacc00e8
> ...
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)