hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-14263) ExploringCompactionPolicy logic around file selection is broken
Date Thu, 20 Aug 2015 00:01:38 GMT
Vladimir Rodionov created HBASE-14263:

             Summary: ExploringCompactionPolicy logic around file selection is broken
                 Key: HBASE-14263
                 URL: https://issues.apache.org/jira/browse/HBASE-14263
             Project: HBase
          Issue Type: Bug
            Reporter: Vladimir Rodionov
            Assignee: Vladimir Rodionov
             Fix For: 2.0.0

It seems that logic around selection of store file candidates is broken:
        // Compute the total size of files that will
        // have to be read if this set of files is compacted.
        long size = getTotalStoreSize(potentialMatchFiles);
        // Store the smallest set of files.  This stored set of files will be used
        // if it looks like the algorithm is stuck.
        if (mightBeStuck && size < smallestSize) {
          smallest = potentialMatchFiles;
          smallestSize = size;
        if (size > comConf.getMaxCompactSize()) {

        if (size >= comConf.getMinCompactSize()
            && !filesInRatio(potentialMatchFiles, currentRatio)) {
This is from applyCompactionPolicy method. As you can see, both min compaction size and max
compaction size are applied to a *selection* of files and not to individual files. It mostly
works as expected only because nobody seems using non-default hbase.hstore.compaction.max.size,
which is  Long.MAX_VALUE  and  it  is not  that  easy  to  figure out  what  is  going  on
 on an opposite side (why small files do not get included?)

This message was sent by Atlassian JIRA

View raw message