hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HBASE-3826) Minor compaction needs to check if still over compactionThreshold after compacting
Date Thu, 28 Apr 2011 21:07:03 GMT

     [ https://issues.apache.org/jira/browse/HBASE-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Nicolas Spiegelberg reassigned HBASE-3826:

    Assignee: Nicolas Spiegelberg

> Minor compaction needs to check if still over compactionThreshold after compacting
> ----------------------------------------------------------------------------------
>                 Key: HBASE-3826
>                 URL: https://issues.apache.org/jira/browse/HBASE-3826
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.90.1
>         Environment: hbase-0.90.1
> hbase-0.90.1-cdh3u0
>            Reporter: Schubert Zhang
>            Assignee: Nicolas Spiegelberg
>              Labels: compaction
>             Fix For: 0.92.0
> I have a busy region, and there are 43 StoreFiles (>compactionThreshold=8) in this
> Now, I stopped the client and stopped putting new data into it. I expect these StoreFiles
to be compacted later.
> But, almost one day later, these 43 StoreFiles are still there.
> (Note: in my hbase instance, I disabled the major compaction.)
> It seems the minor compaction does not be started continuiously to compact remaining
> And I checked the code, it is true.
> -----
> After more test, a obvious issue/problem is, the complete of a minor compaction does
not check if current storefiles need more minor compaction.
> I think this may be a bug or leak.
> Try this test:
> 1. Put many data to a region, then there are 30 storefiles accumulated, because the backend
compaction cannot catch up with the fast puts. (hbase.hstore.compactionThreshold=8, base.hstore.compaction.max=12)
> 2. Then stop put.
> 3. Then, these 30 storefiles are still there for a long time, (no automatic minor compaction)
> 4. Submit a compaction on this region, then, only 12 files are compaction, now, we have
19 storefiles. The minor compaction stopped.
> I think, when a minor compaction complete, it should check if the number of storefiles
still many, if so, another minor compaction should start continuiously.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message