hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Schubert Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-3826) Minor compaction does not be started continuiously to compact remaining storefiles (>compactionThreshold)
Date Thu, 28 Apr 2011 03:54:03 GMT
Minor compaction does not be started continuiously to compact remaining storefiles (>compactionThreshold)
---------------------------------------------------------------------------------------------------------

                 Key: HBASE-3826
                 URL: https://issues.apache.org/jira/browse/HBASE-3826
             Project: HBase
          Issue Type: Improvement
          Components: regionserver
    Affects Versions: 0.90.1
         Environment: hbase-0.90.1
hbase-0.90.1-cdh3u0
            Reporter: Schubert Zhang


I have a busy region, and there are 43 StoreFiles (>compactionThreshold=8) in this region.
Now, I stopped the client and stopped putting new data into it. I expect these StoreFiles
to be compacted later.
 
But, almost one day later, these 43 StoreFiles are still there.
(Note: in my hbase instance, I disabled the major compaction.)
 
It seems the minor compaction does not be started continuiously to compact remaining storefiles.
And I checked the code, it is true.

-----
After more test, a obvious issue/problem is, the complete of a minor compaction does not check
if current storefiles need more minor compaction.
 
I think this may be a bug or leak.
 
Try this test:
 
1. Put many data to a region, then there are 30 storefiles accumulated, because the backend
compaction cannot catch up with the fast puts. (hbase.hstore.compactionThreshold=8, base.hstore.compaction.max=12)
 
2. Then stop put.
 
3. Then, these 30 storefiles are still there for a long time, (no automatic minor compaction)
 
4. Submit a compaction on this region, then, only 12 files are compaction, now, we have 19
storefiles. The minor compaction stopped.
 
I think, when a minor compaction complete, it should check if the number of storefiles still
many, if so, another minor compaction should start continuiously.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message