hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "jiraposter@reviews.apache.org (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4027) Enable direct byte buffers LruBlockCache
Date Fri, 26 Aug 2011 06:53:31 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13091638#comment-13091638
] 

jiraposter@reviews.apache.org commented on HBASE-4027:
------------------------------------------------------



bq.  On 2011-08-26 06:39:36, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/hfile/CacheableDeserializer.java, line
1
bq.  > <https://reviews.apache.org/r/1214/diff/25/?file=34984#file34984line1>
bq.  >
bq.  >     When would we have an instance of Cacheable where we did not need to serialize
AND deserialize?  (Wondering if worth having this extra CacheableDeserializer)

If you wanted something to only reside in the on heap cache, you'd return 0 for getSerializedLength,
and return null for getDeserializer.


bq.  On 2011-08-26 06:39:36, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java, line 44
bq.  > <https://reviews.apache.org/r/1214/diff/25/?file=34983#file34983line44>
bq.  >
bq.  >     Is it ok that "already serialize" and "can't be serialized" return same response
out of this method?

Theres no longer an "already serialized" state. I now remove the object entirely from the
heap. I'll edit the comments.


bq.  On 2011-08-26 06:39:36, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java, line 49
bq.  > <https://reviews.apache.org/r/1214/diff/25/?file=34990#file34990line49>
bq.  >
bq.  >     Whats this mean ('an exception will be thrown..')  Will we ever try to put something
into a block that is larger than the block?  Its a programming error if we do?  If a serialized
object is larger than a 'block', what do we do?

Its a programming error if we do, we shouldn't ever do it, the sizer should make sure of that.
I'll add that to the comments.


bq.  On 2011-08-26 06:39:36, Michael Stack wrote:
bq.  > src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java, line 52
bq.  > <https://reviews.apache.org/r/1214/diff/25/?file=34990#file34990line52>
bq.  >
bq.  >     Where is this ConcurrentLinkedHashMap?  Its not in this class?  I see a ConcurrentMap.
 Its made in the depths of guava?

Its made by Guava's MapMaker. I'll add it to the comments.


- Li


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1214/#review1653
-----------------------------------------------------------


On 2011-08-24 04:08:40, Li Pi wrote:
bq.  
bq.  -----------------------------------------------------------
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/1214/
bq.  -----------------------------------------------------------
bq.  
bq.  (Updated 2011-08-24 04:08:40)
bq.  
bq.  
bq.  Review request for hbase, Todd Lipcon, Ted Yu, Michael Stack, Jonathan Gray, and Li Pi.
bq.  
bq.  
bq.  Summary
bq.  -------
bq.  
bq.  Review request - I apparently can't edit tlipcon's earlier posting of my diff, so creating
a new one.
bq.  
bq.  
bq.  This addresses bug HBase-4027.
bq.      https://issues.apache.org/jira/browse/HBase-4027
bq.  
bq.  
bq.  Diffs
bq.  -----
bq.  
bq.    conf/hbase-env.sh 2d55d27 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/BlockCache.java bfd863e 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/CacheStats.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/Cacheable.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/CacheableDeserializer.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/CachedBlock.java 3b130d8 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/DoubleBlockCache.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/HFileBlock.java 097dc50 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/LruBlockCache.java ae75299 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/SimpleBlockCache.java 7ce4f14 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SingleSizeCache.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/slab/Slab.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabCache.java PRE-CREATION 
bq.    src/main/java/org/apache/hadoop/hbase/io/hfile/slab/SlabItemEvictionWatcher.java PRE-CREATION

bq.    src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java 3efba7f 
bq.    src/main/java/org/apache/hadoop/hbase/regionserver/StoreFile.java 7b7bf73 
bq.    src/main/java/org/apache/hadoop/hbase/util/DirectMemoryUtils.java PRE-CREATION 
bq.    src/test/java/org/apache/hadoop/hbase/io/hfile/CacheTestUtils.java PRE-CREATION 
bq.    src/test/java/org/apache/hadoop/hbase/io/hfile/TestCachedBlockQueue.java 1ad2ece 
bq.    src/test/java/org/apache/hadoop/hbase/io/hfile/TestLruBlockCache.java f0a9832 
bq.    src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSingleSizeCache.java PRE-CREATION

bq.    src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlab.java PRE-CREATION 
bq.    src/test/java/org/apache/hadoop/hbase/io/hfile/slab/TestSlabCache.java PRE-CREATION

bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestMemStoreLAB.java d7e43a0 
bq.    src/test/java/org/apache/hadoop/hbase/regionserver/TestStoreFile.java 4387170 
bq.  
bq.  Diff: https://reviews.apache.org/r/1214/diff
bq.  
bq.  
bq.  Testing
bq.  -------
bq.  
bq.  Ran benchmarks against it in HBase standalone mode. Wrote test cases for all classes,
multithreaded test cases exist for the cache.
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Li
bq.  
bq.



> Enable direct byte buffers LruBlockCache
> ----------------------------------------
>
>                 Key: HBASE-4027
>                 URL: https://issues.apache.org/jira/browse/HBASE-4027
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jason Rutherglen
>            Assignee: Li Pi
>            Priority: Minor
>         Attachments: 4027-v5.diff, 4027v7.diff, HBase-4027 (1).pdf, HBase-4027.pdf, HBase4027v8.diff,
HBase4027v9.diff, hbase-4027-v10.5.diff, hbase-4027-v10.diff, hbase-4027v10.6.diff, hbase-4027v13.1.diff,
hbase-4027v6.diff, hbase4027v11.5.diff, hbase4027v11.6.diff, hbase4027v11.7.diff, hbase4027v11.diff,
hbase4027v12.1.diff, hbase4027v12.diff, hbase4027v15.2.diff, slabcachepatch.diff, slabcachepatchv2.diff,
slabcachepatchv3.1.diff, slabcachepatchv3.2.diff, slabcachepatchv3.diff, slabcachepatchv4.5.diff,
slabcachepatchv4.diff
>
>
> Java offers the creation of direct byte buffers which are allocated outside of the heap.
> They need to be manually free'd, which can be accomplished using an documented {{clean}}
method.
> The feature will be optional.  After implementing, we can benchmark for differences in
speed and garbage collection observances.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message