hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan Rawson" <ryano...@gmail.com>
Subject Re: Review Request: Add option to cache blocks on hfile write and evict blocks on hfile close
Date Tue, 30 Nov 2010 17:57:27 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/1261/#review2010
-----------------------------------------------------------



branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java
<http://review.cloudera.org/r/1261/#comment6349>

    why would you not want to evict blocks from the cache on close?


- Ryan


On 2010-11-29 23:22:38, Jonathan Gray wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> http://review.cloudera.org/r/1261/
> -----------------------------------------------------------
> 
> (Updated 2010-11-29 23:22:38)
> 
> 
> Review request for hbase, stack and khemani.
> 
> 
> Summary
> -------
> 
> This issue is about adding configuration options to add/remove from the block cache when
creating/closing files. For use cases with lots of flushing and compacting, this might be
desirable to prevent cache misses and maximize the effective utilization of total block cache
capacity.
> 
> The first option, hbase.rs.cacheblocksonwrite, will make it so we pre-cache blocks as
we are writing out new files.
> 
> The second option, hbase.rs.evictblocksonclose, will make it so we evict blocks when
files are closed.
> 
> 
> This addresses bug HBASE-3287.
>     http://issues.apache.org/jira/browse/HBASE-3287
> 
> 
> Diffs
> -----
> 
>   branches/0.90/src/main/java/org/apache/hadoop/hbase/io/HalfStoreFileReader.java 1040422

>   branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java 1040422

>   branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/HFile.java 1040422 
>   branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java 1040422

>   branches/0.90/src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java
1040422 
>   branches/0.90/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java
1040422 
>   branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/Store.java 1040422

>   branches/0.90/src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 1040422

>   branches/0.90/src/main/java/org/apache/hadoop/hbase/util/CompressionTest.java 1040422

>   branches/0.90/src/test/java/org/apache/hadoop/hbase/HFilePerformanceEvaluation.java
1040422 
>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/TestHalfStoreFileReader.java
1040422 
>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/RandomSeek.java 1040422

>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFile.java 1040422

>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFilePerformance.java
1040422 
>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestHFileSeek.java 1040422

>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestReseekTo.java 1040422

>   branches/0.90/src/test/java/org/apache/hadoop/hbase/io/hfile/TestSeekTo.java 1040422

>   branches/0.90/src/test/java/org/apache/hadoop/hbase/mapreduce/TestLoadIncrementalHFiles.java
1040422 
>   branches/0.90/src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java
1040422 
> 
> Diff: http://review.cloudera.org/r/1261/diff
> 
> 
> Testing
> -------
> 
> Added a unit test to TestStoreFile.  That passes.
> 
> Need to do perf testing on a cluster.
> 
> 
> Thanks,
> 
> Jonathan
> 
>


Mime
View raw message