hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Neal Young (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-16876) RatioBasedCompactionPolicy ignores mayBeStuck
Date Tue, 18 Oct 2016 22:34:58 GMT

     [ https://issues.apache.org/jira/browse/HBASE-16876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Neal Young updated HBASE-16876:
-------------------------------
    Description: 
I'm a newbie so may not be reporting this bug correctly.  The bug currently shows in lines
181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181
.
{code:title=Bar.java|borderStyle=solid}
176     while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
177       fileSizes[start] > Math.max(comConf.getMinCompactSize(),
178           (long) (sumSize[start + 1] * ratio))) {
179       ++start;
180     }
181     if (start < countOfFiles) {
182       LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
183         + " files from " + countOfFiles + " candidates");
184     } else if (mayBeStuck) {
185       // We may be stuck. Compact the latest files if we can.
186       int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
187       if (filesToLeave >= 0) {
188         start = filesToLeave;
189       }
190     }
{code}

On reaching line 176, start = 0.  When comConf.getMinFilesToCompact() is at least 2 (as occurs
in the default), the while loop is guaranteed to terminate with start < countOfFiles. 
Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.

Perhaps the following code would be better, but I'm not sure:

{code:title=Bar.java|borderStyle=solid}
    while (start < countOfFiles
           && fileSizes[start] > Math.max(comConf.getMinCompactSize(),
                                          (long) (sumSize[start + 1] * ratio))) {
        ++start;
    }
    if (countOfFiles - start < comConf.getMinFilesToCompact()) {
        if (mayBeStuck && countOfFiles >= comConf.getMinFilesToCompact()) {
            start = countOfFiles - comConf.getMinFilesToCompact();
        } else {
            start = countOfFiles;
        }
    }
    if (countOfFiles - start > 0) {
        LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
                 + " files from " + countOfFiles + " candidates");
    }
{code}


  was:
I'm a newbie so may not be reporting this bug correctly.  The bug currently shows in lines
181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181
.
{code:title=Bar.java|borderStyle=solid}
176     while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
177       fileSizes[start] > Math.max(comConf.getMinCompactSize(),
178           (long) (sumSize[start + 1] * ratio))) {
179       ++start;
180     }
181     if (start < countOfFiles) {
182       LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
183         + " files from " + countOfFiles + " candidates");
184     } else if (mayBeStuck) {
185       // We may be stuck. Compact the latest files if we can.
186       int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
187       if (filesToLeave >= 0) {
188         start = filesToLeave;
189       }
190     }
{code}

On reaching line 176, start = 0.  When comConf.getMinFilesToCompact() is at least 2 (as occurs
in the default), the while loop is guaranteed to terminate with start < countOfFiles. 
Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.

Perhaps the following code would be better, but I'm not sure:

{code:title=Bar.java|borderStyle=solid}
    while (start < countOfFiles
           && fileSizes[start] > Math.max(comConf.getMinCompactSize(),
                                          (long) (sumSize[start + 1] * ratio))) {
        ++start;
    }
    if (start < countOfFiles) {
        if (countOfFiles - start >= comConf.getMinFilesToCompact()) {
            LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
                     + " files from " + countOfFiles + " candidates");
        } else {
            start = countOfFiles;
        }
    } else if (mayBeStuck) {
        // We may be stuck. Compact the latest files if we can.
        int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
        if (filesToLeave >= 0) {
            start = filesToLeave;
        }
    }
{code}



> RatioBasedCompactionPolicy ignores mayBeStuck
> ---------------------------------------------
>
>                 Key: HBASE-16876
>                 URL: https://issues.apache.org/jira/browse/HBASE-16876
>             Project: HBase
>          Issue Type: Bug
>          Components: Compaction
>            Reporter: Neal Young
>              Labels: newbie
>
> I'm a newbie so may not be reporting this bug correctly.  The bug currently shows in
lines 181 - 190 here : http://hbase.apache.org/xref/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.html#181
.
> {code:title=Bar.java|borderStyle=solid}
> 176     while (countOfFiles - start >= comConf.getMinFilesToCompact() &&
> 177       fileSizes[start] > Math.max(comConf.getMinCompactSize(),
> 178           (long) (sumSize[start + 1] * ratio))) {
> 179       ++start;
> 180     }
> 181     if (start < countOfFiles) {
> 182       LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
> 183         + " files from " + countOfFiles + " candidates");
> 184     } else if (mayBeStuck) {
> 185       // We may be stuck. Compact the latest files if we can.
> 186       int filesToLeave = candidates.size() - comConf.getMinFilesToCompact();
> 187       if (filesToLeave >= 0) {
> 188         start = filesToLeave;
> 189       }
> 190     }
> {code}
> On reaching line 176, start = 0.  When comConf.getMinFilesToCompact() is at least 2 (as
occurs in the default), the while loop is guaranteed to terminate with start < countOfFiles.
 Hence, the else clause starting at line 184 never executes, regardless of the value of mayBeStuck.
> Perhaps the following code would be better, but I'm not sure:
> {code:title=Bar.java|borderStyle=solid}
>     while (start < countOfFiles
>            && fileSizes[start] > Math.max(comConf.getMinCompactSize(),
>                                           (long) (sumSize[start + 1] * ratio))) {
>         ++start;
>     }
>     if (countOfFiles - start < comConf.getMinFilesToCompact()) {
>         if (mayBeStuck && countOfFiles >= comConf.getMinFilesToCompact())
{
>             start = countOfFiles - comConf.getMinFilesToCompact();
>         } else {
>             start = countOfFiles;
>         }
>     }
>     if (countOfFiles - start > 0) {
>         LOG.info("Default compaction algorithm has selected " + (countOfFiles - start)
>                  + " files from " + countOfFiles + " candidates");
>     }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message