hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Rutherglen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4027) Enable direct byte buffers LruBlockCache
Date Fri, 24 Jun 2011 23:22:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13054747#comment-13054747
] 

Jason Rutherglen commented on HBASE-4027:
-----------------------------------------

{quote}This would be really useful. I think even making copies out of the direct byte buffers
would confer a substantial performance advantage over the FS cache.{quote}

The filesystem cache doesn't help because HBase needs quick access to uncompressed blocks
for scanning.  For what duration does an uncompressed block need to be cached?  In either
case, accessing compressed blocks from the FS cache will be faster than hitting the disk or
network.  I am guessing one can maintain a small'ish block cache, ensure HDFS blocks are local,
provide extra space for the FS cache, and gain in read throughput.  

Snappy should decompress fast enough for this to be more viable than maintaining a large-ish
block cache.  The problem [today] with a small'ish block cache is the GC is driven mad.


> Enable direct byte buffers LruBlockCache
> ----------------------------------------
>
>                 Key: HBASE-4027
>                 URL: https://issues.apache.org/jira/browse/HBASE-4027
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Jason Rutherglen
>            Priority: Minor
>
> Java offers the creation of direct byte buffers which are allocated outside of the heap.
> They need to be manually free'd, which can be accomplished using an documented {{clean}}
method.
> The feature will be optional.  After implementing, we can benchmark for differences in
speed and garbage collection observances.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message