hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19199) RatioBasedCompactionPolicy#shouldPerformMajorCompaction() always return true when only one file needs to compact
Date Fri, 10 Nov 2017 06:23:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16247068#comment-16247068
] 

Hudson commented on HBASE-19199:
--------------------------------

FAILURE: Integrated in Jenkins build HBase-2.0 #828 (See [https://builds.apache.org/job/HBase-2.0/828/])
HBASE-19199 RatioBasedCompactionPolicy#shouldPerformMajorCompaction() (tedyu: rev b6b4cd40cd3b81147fd164ab5337587c91954174)
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/compactions/RatioBasedCompactionPolicy.java


> RatioBasedCompactionPolicy#shouldPerformMajorCompaction() always return true when only
one file needs to compact
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-19199
>                 URL: https://issues.apache.org/jira/browse/HBASE-19199
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 1.3.1, 1.4.1, 1.5.0, 2.0.0-beta-1
>            Reporter: Guangxu Cheng
>            Assignee: Guangxu Cheng
>             Fix For: 2.0.0-beta-1
>
>         Attachments: HBASE-19199.patch, HBASE-19199.patch, HBASE-19199.patch
>
>
> {code:title=RatioBasedCompactionPolicy.java|borderStyle=solid}
>   public boolean shouldPerformMajorCompaction(Collection<HStoreFile> filesToCompact)
>     throws IOException {
>     boolean result = false;
>     long mcTime = getNextMajorCompactTime(filesToCompact);
>     if (filesToCompact == null || filesToCompact.isEmpty() || mcTime == 0) {
>       return result;
>     }
>     // TODO: Use better method for determining stamp of last major (HBASE-2990)
>     long lowTimestamp = StoreUtils.getLowestTimestamp(filesToCompact);
>     long now = EnvironmentEdgeManager.currentTime();
>     if (lowTimestamp > 0L && lowTimestamp < (now - mcTime)) {
>       String regionInfo;
>       if (this.storeConfigInfo != null && this.storeConfigInfo instanceof HStore)
{
>         regionInfo = ((HStore)this.storeConfigInfo).getRegionInfo().getRegionNameAsString();
>       } else {
>         regionInfo = this.toString();
>       }
>       // Major compaction time has elapsed.
>       long cfTTL = HConstants.FOREVER;
>       if (this.storeConfigInfo != null) {
>          cfTTL = this.storeConfigInfo.getStoreFileTtl();
>       }
>       if (filesToCompact.size() == 1) {
>         // Single file
>         HStoreFile sf = filesToCompact.iterator().next();
>         OptionalLong minTimestamp = sf.getMinimumTimestamp();
>         long oldest = minTimestamp.isPresent() ? now - minTimestamp.getAsLong() : Long.MIN_VALUE;
>         if (sf.isMajorCompactionResult() && (cfTTL == Long.MAX_VALUE || oldest
< cfTTL)) {
>           float blockLocalityIndex =
>             sf.getHDFSBlockDistribution().getBlockLocalityIndex(
>             RSRpcServices.getHostname(comConf.conf, false));
>           if (blockLocalityIndex < comConf.getMinLocalityToForceCompact()) {
>             LOG.debug("Major compaction triggered on only store " + regionInfo
>               + "; to make hdfs blocks local, current blockLocalityIndex is "
>               + blockLocalityIndex + " (min " + comConf.getMinLocalityToForceCompact()
+ ")");
>             result = true;
>           } else {
>             LOG.debug("Skipping major compaction of " + regionInfo
>               + " because one (major) compacted file only, oldestTime " + oldest
>               + "ms is < TTL=" + cfTTL + " and blockLocalityIndex is " + blockLocalityIndex
>               + " (min " + comConf.getMinLocalityToForceCompact() + ")");
>           }
>         } else if (cfTTL != HConstants.FOREVER && oldest > cfTTL) {
>           LOG.debug("Major compaction triggered on store " + regionInfo
>             + ", because keyvalues outdated; time since last major compaction "
>             + (now - lowTimestamp) + "ms");
>           result = true;
>         }
>       } else {
>         LOG.debug("Major compaction triggered on store " + regionInfo
>           + "; time since last major compaction " + (now - lowTimestamp) + "ms");
>       }
>       result = true;
>     }
>     return result;
>   }
> {code}
> {code}  
>    } else {
>         LOG.debug("Major compaction triggered on store " + regionInfo
>           + "; time since last major compaction " + (now - lowTimestamp) + "ms");
>       }
>       result = true;
> {code}
> if (lowTimestamp > 0L && lowTimestamp < (now - mcTime)), the result always
equal true.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message