Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 28 Apr 2011 19:24:03 +0000 (UTC)
From: "Jean-Daniel Cryans (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <96333233.9635.1304018643146.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1734770722.7660.1303962843346.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Updated] (HBASE-3826) Minor compaction needs to check if
 still over compactionThreshold after compacting
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HBASE-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jean-Daniel Cryans updated HBASE-3826:
--------------------------------------

    Fix Version/s: 0.92.0
          Summary: Minor compaction needs to check if still over compactionThreshold after compacting  (was: Minor compaction does not be started continuiously to compact remaining storefiles (>compactionThreshold))

Editing the title and marking against 0.92

> Minor compaction needs to check if still over compactionThreshold after compacting
> ----------------------------------------------------------------------------------
>
>                 Key: HBASE-3826
>                 URL: https://issues.apache.org/jira/browse/HBASE-3826
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.90.1
>         Environment: hbase-0.90.1
> hbase-0.90.1-cdh3u0
>            Reporter: Schubert Zhang
>              Labels: compaction
>             Fix For: 0.92.0
>
>
> I have a busy region, and there are 43 StoreFiles (>compactionThreshold=8) in this region.
> Now, I stopped the client and stopped putting new data into it. I expect these StoreFiles to be compacted later.
>  
> But, almost one day later, these 43 StoreFiles are still there.
> (Note: in my hbase instance, I disabled the major compaction.)
>  
> It seems the minor compaction does not be started continuiously to compact remaining storefiles.
> And I checked the code, it is true.
> -----
> After more test, a obvious issue/problem is, the complete of a minor compaction does not check if current storefiles need more minor compaction.
>  
> I think this may be a bug or leak.
>  
> Try this test:
>  
> 1. Put many data to a region, then there are 30 storefiles accumulated, because the backend compaction cannot catch up with the fast puts. (hbase.hstore.compactionThreshold=8, base.hstore.compaction.max=12)
>  
> 2. Then stop put.
>  
> 3. Then, these 30 storefiles are still there for a long time, (no automatic minor compaction)
>  
> 4. Submit a compaction on this region, then, only 12 files are compaction, now, we have 19 storefiles. The minor compaction stopped.
>  
> I think, when a minor compaction complete, it should check if the number of storefiles still many, if so, another minor compaction should start continuiously.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira