hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Phabricator (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-5010) Filter HFiles based on TTL
Date Sun, 25 Dec 2011 23:36:32 GMT

     [ https://issues.apache.org/jira/browse/HBASE-5010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Phabricator updated HBASE-5010:
-------------------------------

    Attachment: D1017.1.patch

mbautin requested code review of "[jira] [HBASE-5010] Filter HFiles based on TTL".
Reviewers: Kannan, Liyin, tedyu, JIRA

  This is the trunk version of D909. The main difference is that there is a minVersions CF
setting in trunk, and when minVersions is not zero, we can't exclude StoreFiles based on TTL,
because we might have to retrieve KVs with expired timestamps to comply with the minVersions
requirement.

TEST PLAN
  Unit tests (including a new one).

REVISION DETAIL
  https://reviews.facebook.net/D1017

AFFECTED FILES
  src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java
  src/main/java/org/apache/hadoop/hbase/regionserver/ExplicitColumnTracker.java
  src/main/java/org/apache/hadoop/hbase/regionserver/KeyValueScanner.java
  src/main/java/org/apache/hadoop/hbase/regionserver/MemStore.java
  src/main/java/org/apache/hadoop/hbase/regionserver/NonLazyKeyValueScanner.java
  src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java
  src/main/java/org/apache/hadoop/hbase/regionserver/ScanWildcardColumnTracker.java
  src/main/java/org/apache/hadoop/hbase/regionserver/Store.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreFileScanner.java
  src/main/java/org/apache/hadoop/hbase/regionserver/StoreScanner.java
  src/main/java/org/apache/hadoop/hbase/regionserver/TimeRangeTracker.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompaction.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestCompoundBloomFilter.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestExplicitColumnTracker.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStore.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestMinVersions.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestQueryMatcher.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestScanWildcardColumnTracker.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestSelectScannersUsingTTL.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
  src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreScanner.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/2127/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.

                
> Filter HFiles based on TTL
> --------------------------
>
>                 Key: HBASE-5010
>                 URL: https://issues.apache.org/jira/browse/HBASE-5010
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Mikhail Bautin
>            Assignee: Mikhail Bautin
>         Attachments: D1017.1.patch, D909.1.patch, D909.2.patch
>
>
> In ScanWildcardColumnTracker we have
> {code:java}
>  
>   this.oldestStamp = EnvironmentEdgeManager.currentTimeMillis() - ttl;
>   ...
>   private boolean isExpired(long timestamp) {
>     return timestamp < oldestStamp;
>   }
> {code}
> but this time range filtering does not participate in HFile selection. In one real case
this caused next() calls to time out because all KVs in a table got expired, but next() had
to iterate over the whole table to find that out. We should be able to filter out those HFiles
right away. I think a reasonable approach is to add a "default timerange filter" to every
scan for a CF with a finite TTL and utilize existing filtering in StoreFile.Reader.passesTimerangeFilter.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message