hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (HBASE-4717) More efficient age-off of old data during major compaction
Date Sat, 11 Apr 2015 01:17:13 GMT

     [ https://issues.apache.org/jira/browse/HBASE-4717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Purtell resolved HBASE-4717.
    Resolution: Incomplete

> More efficient age-off of old data during major compaction
> ----------------------------------------------------------
>                 Key: HBASE-4717
>                 URL: https://issues.apache.org/jira/browse/HBASE-4717
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>    Affects Versions: 0.94.0
>            Reporter: Todd Lipcon
> Many applications need to implement efficient age-off of old data. We currently only
perform age-off during major compaction by scanning through all of the KVs. Instead, we could
implement the following:
> - Set hbase.hstore.compaction.max.size reasonably small. Thus, older store files contain
only smaller finite ranges of time.
> - Periodically run an "age-off compaction". This compaction would scan the current list
of storefiles. Any store file that falls entirely out of the TTL time range would be dropped.
Store files completely within the time range would be un-altered. Those crossing the time-range
boundary could either be left alone or compacted using the existing compaction code.
> I don't have a design in mind for how exactly this would be implemented, but hope to
generate some discussion.

This message was sent by Atlassian JIRA

View raw message