hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4018) Attach memcached as secondary block cache to regionserver
Date Wed, 22 Jun 2011 21:44:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13053486#comment-13053486
] 

Jonathan Gray commented on HBASE-4018:
--------------------------------------

bq. Optimal solution would be building a slab allocated block cache within java. Use reference
counting for a zero copy solution. This is difficult to implement and debug though.

I'm working on this.  I think implementing both directions is worthwhile and we can run good
comparisons (including against linux fs cache + local datanodes).

bq. It would seem best to move in the direction of local HDFS file access and allow plugging
in the block cache as a point of comparison / legacy.

I think it's best to move in all directions and do comparisons.  I've already seen performance
differences between fs cache and the actual hbase block cache.  There's also compressed vs.
decompressed (fs cache will always be compressed)

> Attach memcached as secondary block cache to regionserver
> ---------------------------------------------------------
>
>                 Key: HBASE-4018
>                 URL: https://issues.apache.org/jira/browse/HBASE-4018
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Li Pi
>            Assignee: Li Pi
>
> Currently, block caches are limited by heap size, which is limited by garbage collection
times in Java.
> We can get around this by using memcached w/JNI as a secondary block cache. This should
be faster than the linux file system's caching, and allow us to very quickly gain access to
a high quality slab allocated cache.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message