hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Schubert Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-3826) Minor compaction needs to check if still over compactionThreshold after compacting
Date Sat, 30 Apr 2011 11:02:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13027318#comment-13027318

Schubert Zhang commented on HBASE-3826:

The patch seems re-queue compaction at reigion level.
How about in store.

> Minor compaction needs to check if still over compactionThreshold after compacting
> ----------------------------------------------------------------------------------
>                 Key: HBASE-3826
>                 URL: https://issues.apache.org/jira/browse/HBASE-3826
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.90.1
>         Environment: hbase-0.90.1
> hbase-0.90.1-cdh3u0
>            Reporter: Schubert Zhang
>            Assignee: Nicolas Spiegelberg
>              Labels: compaction
>             Fix For: 0.92.0
>         Attachments: HBASE-3826.patch
> I have a busy region, and there are 43 StoreFiles (>compactionThreshold=8) in this
> Now, I stopped the client and stopped putting new data into it. I expect these StoreFiles
to be compacted later.
> But, almost one day later, these 43 StoreFiles are still there.
> (Note: in my hbase instance, I disabled the major compaction.)
> It seems the minor compaction does not be started continuiously to compact remaining
> And I checked the code, it is true.
> -----
> After more test, a obvious issue/problem is, the complete of a minor compaction does
not check if current storefiles need more minor compaction.
> I think this may be a bug or leak.
> Try this test:
> 1. Put many data to a region, then there are 30 storefiles accumulated, because the backend
compaction cannot catch up with the fast puts. (hbase.hstore.compactionThreshold=8, base.hstore.compaction.max=12)
> 2. Then stop put.
> 3. Then, these 30 storefiles are still there for a long time, (no automatic minor compaction)
> 4. Submit a compaction on this region, then, only 12 files are compaction, now, we have
19 storefiles. The minor compaction stopped.
> I think, when a minor compaction complete, it should check if the number of storefiles
still many, if so, another minor compaction should start continuiously.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message