hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "HBase Review Board (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3287) Add option to cache blocks on hfile write and evict blocks on hfile close
Date Tue, 30 Nov 2010 07:25:14 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965134#action_12965134
] 

HBase Review Board commented on HBASE-3287:
-------------------------------------------

Message from: "Jonathan Gray" <jgray@apache.org>

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1261/
-----------------------------------------------------------

Review request for hbase, stack and khemani.


Summary
-------

This issue is about adding configuration options to add/remove from the block cache when creating/closing
files. For use cases with lots of flushing and compacting, this might be desirable to prevent
cache misses and maximize the effective utilization of total block cache capacity.

The first option, hbase.rs.cacheblocksonwrite, will make it so we pre-cache blocks as we are
writing out new files.

The second option, hbase.rs.evictblocksonclose, will make it so we evict blocks when files
are closed.


This addresses bug HBASE-3287.
    http://issues.apache.org/jira/browse/HBASE-3287


Diffs
-----

  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java
1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 1040422


Diff: http://review.cloudera.org/r/1261/diff


Testing
-------

Added a unit test to TestStoreFile.  That passes.

Need to do perf testing on a cluster.


Thanks,

Jonathan




> Add option to cache blocks on hfile write and evict blocks on hfile close
> -------------------------------------------------------------------------
>
>                 Key: HBASE-3287
>                 URL: https://issues.apache.org/jira/browse/HBASE-3287
>             Project: HBase
>          Issue Type: Improvement
>          Components: io, regionserver
>    Affects Versions: 0.90.0
>            Reporter: Jonathan Gray
>            Assignee: Jonathan Gray
>             Fix For: 0.92.0
>
>
> This issue is about adding configuration options to add/remove from the block cache when
creating/closing files.  For use cases with lots of flushing and compacting, this might be
desirable to prevent cache misses and maximize the effective utilization of total block cache
capacity.
> The first option, {{hbase.rs.cacheblocksonwrite}}, will make it so we pre-cache blocks
as we are writing out new files.
> The second option, {{hbase.rs.evictblocksonclose}}, will make it so we evict blocks when
files are closed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message