hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray" <jg...@apache.org>
Subject Review Request: Add option to cache blocks on hfile write and evict blocks on hfile close
Date Tue, 30 Nov 2010 07:22:38 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1261/
-----------------------------------------------------------

Review request for hbase, stack and khemani.


Summary
-------

This issue is about adding configuration options to add/remove from the block cache when creating/closing
files. For use cases with lots of flushing and compacting, this might be desirable to prevent
cache misses and maximize the effective utilization of total block cache capacity.

The first option, hbase.rs.cacheblocksonwrite, will make it so we pre-cache blocks as we are
writing out new files.

The second option, hbase.rs.evictblocksonclose, will make it so we evict blocks when files
are closed.


This addresses bug HBASE-3287.
    http://issues.apache.org/jira/browse/HBASE-3287


Diffs
-----

  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1040422 
  branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1040422

  branches/0.90/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 1040422

  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java
1040422 
  branches/0.90/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 1040422


Diff: http://review.cloudera.org/r/1261/diff


Testing
-------

Added a unit test to TestStoreFile.  That passes.

Need to do perf testing on a cluster.


Thanks,

Jonathan


Mime
View raw message