hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15055) Major compaction is not triggered when both of TTL and hbase.hstore.compaction.max.size are set
Date Mon, 11 Jan 2016 19:21:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15092520#comment-15092520
] 

Jerry He commented on HBASE-15055:
----------------------------------

Good summary from Anoop.

bq. 1. When this max size is specified, at least the major compaction should select all files
and do a real major compaction right? Else when those will get compacted? Possible tomb stone
removal! This should happen irrespective of the TTL thing no?

In the current code, it seems the skipLargeFiles() is possibly not called if 'forceMajor'
is true.
{code}
    if (!(forceMajor && isAllFiles)) {
      candidateSelection = skipLargeFiles(candidateSelection, mayUseOffPeak);
      isAllFiles = candidateFiles.size() == candidateSelection.size();
    }
{code}

In the ExploringCompactionPolicy, (which is the default now), there is another place to check
the maxCompactSize in applyCompactionPolicy()
{code}
if (size > comConf.getMaxCompactSize(mayUseOffPeak)) {
          continue;
        }
{code}


> Major compaction is not triggered when both of TTL and hbase.hstore.compaction.max.size
are set
> -----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-15055
>                 URL: https://issues.apache.org/jira/browse/HBASE-15055
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Eungsop Yoo
>            Assignee: Eungsop Yoo
>            Priority: Minor
>         Attachments: HBASE-15055-v1.patch, HBASE-15055-v2.patch, HBASE-15055-v3.patch,
HBASE-15055-v4.patch, HBASE-15055-v5.patch, HBASE-15055-v6.patch, HBASE-15055-v7.patch, HBASE-15055-v8.patch,
HBASE-15055-v9.patch, HBASE-15055.patch
>
>
> Some large files may be skipped by hbase.hstore.compaction.max.size in candidate selection.
It causes skipping of major compaction. So the TTL expired records are still remained in the
disks and keep consuming disks.
> To resolve this issue, I suggest that to skip large files only if there is no TTL expired
record.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message