hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clara Xiong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15339) Improve DateTieredCompactionPolicy
Date Tue, 29 Mar 2016 02:36:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215300#comment-15215300
] 

Clara Xiong commented on HBASE-15339:
-------------------------------------

maxAge, in short words, is the configuration to control how far you go when promoting tiers.
Once a window is out of this range, say more than a year old from current time, we don't combine
the windows to promote to the next tiers any more. It doesn't mean they are out of compaction
control. Within DTC, minor compaction won't touch these files any more but major compaction
will still touch them. It actually makes TTL or archiving much simpler and more efficient
since any files older than max age but younger than TTL (or no TTL) will not be combined(I
used the term "compacted" but this might confusing) into larger sizes.This way, TTL and archiving
process can just scan from oldest files to remove/archive files based on their time range
instead of scanning a very large file  from major compaction by Ratio-based compaction.

The design here is similar to FIFOCompactionPolicy you described.


> Improve DateTieredCompactionPolicy
> ----------------------------------
>
>                 Key: HBASE-15339
>                 URL: https://issues.apache.org/jira/browse/HBASE-15339
>             Project: HBase
>          Issue Type: Improvement
>          Components: Compaction
>            Reporter: Duo Zhang
>
> Add multi-output support.
> Add archive old data support.
> ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message